The quality of talks were all superb, indeed. Probably over half of the presenters are either principle developers on high profile projects or they have written a book or own their own consulting company. On day zero, where there were 3hr long tutorial sessions, I spend the morning in Mark Ramm’s TurboGears but then I switched over to the PHP side in the afternoon to catch Scott MacVicar and Helgi Þormar Þorbjörnsson’s Caching for Cash.

At the start of day one, the first day of the normal sessions, I think everyone was expecting a lot more people. There were, in fact, more people but not as many as I was expecting, but again that’s perfectly okay. This day was a full one, starting off with the keynote by Kevin Dangoor about Growing your Community. After a break I then attended Decorators are Fun by Matt Wilson and learned that he is not that far away from me in Cleveland. Next I attended another Mark Ramm talk about WSGI where he was explaining how easy it was to build a web framework. It was given a bit “tongue in check” since he is the primary maintainer of TurboGears. Following that, I attended a middle track session about Distributed version control with GIT by Travis Swicegood. Travis had just finished writing a book about using GIT called Pragmatic Version Control Using Git and not surprisingly gave a authoritation explanation of using GIT. Following lunch, I attending another PHP track presentation but it could have been in the neutral middle track. The talk was Map, Filter, Reduce In the Small and in the Cloud by Sebastian Bergmann where he explained the popular functional programming techniques popularized by Google for computing large quantities of data. Sebastian gave me another reason to checkout Hadoop and in fact I’m now thinking of another Python Magazine article about using hadoop with Jython. For the last session of the day I decided to attend Michael Foord’s talk about IronPython. I didn’t think I’d ever checkout IronPython on my own, so I thought I’d get a crash course from Michael who also just finished work on his book IronPython in Action.

Still not done with day one. After all of the normal presentation’s concluded, we had happy hour while gearing up for the Pecha Kucha competition sessions. Pecha Kucha is where you provide 20 slides and set them to auto switch every 20 seconds making your session a little over six minutes. Apparently people have found that you can get the same quality bits of information in that format as compared to a full hour session. At least that is what the Japanese have concluded. As for PHP/PyWorks, we mostly had fun with the sessions. There were talks about web security, general ranting, LOLCode, and many others which I’m having a problem remembering. At the end, the LOLCode talk took the prize of the Xbox 360 gaming system by our judges and if you’d really like to see what was going on, you may be able to watch streamed video captured by Travis Swicegood’s iPhone. Before I went to bed, I rehearsed my presentation one more time.

By the time day two started, it felt like I had been there a full week and yet we still had a full day of presentations again. I started the morning in Chris Perkins’s talk about the Sphinx Documentation System. We all understand the importance of documentation and it’s not always fun, but again I thought investing 45min catching up on some of the Python “best practices” for documentation would be well worth the time. Afterwards, I stayed in the same room for Jacob Taylor’s talk about Exploring Artificial Intelligence with Python. Jacob didn’t get around to showing any Python code but he had good attendance for being a founder of SugarCRM. Next, the highlight of the conference, my presentation about LDAP and Python. The number of attendees for my presentation were average for the Python sessions and by this point I felt like I knew everyone which removed any pressure or nervousness. We’ll see how interested people were by seeing who downloads my configparser.py and/or ldapconfig.py scripts. After lunch, I attended Kevin Dangoor’s Paver talk where he explained the motivations for Paver and showed numerous examples of what pain points it solves. Finally, the last session I attended at PyWorks was Jonathan LaCour’s talk about Elixir, the Python module which makes introduction into SQLAlchemy an easy one. Elixir helps kick start your DB code by simplifying SQLAlchemy by making a lot of sane choices for you as well as providing other conveniences. Jonathan had to work hard to get all of his content into his hour, mostly because he gave a decent overview of SQLAlchemy and then his Elixir module.

As with the previous day, this day concluded with another happy hour while waiting for our closing keynote. The closing keynote was given by Jay Pipes about “living in the gray areas” and not sticking to extreme black and white of our technologies. He praised the joint efforts being made by the PHP and Python folks and criticized people who are too biased to learn from the other communities. Jay is working on Drizzle, while working for Sun, where they are challanging all of the preconceived notions being made by the MySQL community. Drizzle is basically a fork of MySQL and their goals are to provide a much more streamlined version of a database. Jay explained that forks are good (as well as “sporks”) because it keeps people on their toes and keeps the level of competition up. Finally, Jay’s last point was that we need to spend more time listening to other people and less time preaching our biased opinions.

I overheard PHP and Python people resonating Jay’s message after the keynote. I’m glad to have participated in such a successful conference where I truely believe boundries were crossed. With as much time that I spend with the PHP folks, I was repeatedly asked, “So, you coming over to the PHP side?” I think the last time I was asked that was in the hotel pool where again I was playing the role of the “token Python guy” amongst the PHP folks. To be honest, those PHP folks know how to have fun, and if my criteria for choosing a programming language was the amount of fun the community had I would be doing PHP development. I definately want attend next year’s PyWorks and PHP conference and I have an entire year to come up with my presentation proposals.

A Google developer, working on the zxing project (pronounced “zebra crossing”), has printed a 2D barcode encoding of his personal information on the back of his business card. With the builtin camera, on his android phone, he can scan in a barcode and immediate use the encoded data. It is an impressive demonstration of integrating technology with our mobile devices. Check out the video which has inspired me to not only do the same but also write this small informational note about 2D barcodes.

If you didn’t catch it, the format of the 2D barcode on his business card is QR Code. Among the other 2D barcode formats, QR Code barcodes are most popular in Japan where it was invented by Denso-Wave. The popularity of QR Codes in Japan has grown to the level of being supported by nearly every mobile device there and that also means finding QR Codes available on a increasing amount of printed media from fliers to magazines and coupons.

There are other competing 2D barcode formats that I could choose from but after doing researching and not seeing any distinctive advantages, I have concluded to follow suit with the google developer and hope that the android phone’s application and popularity will help propel QR Code’s popularity over the other 2D barcode formats.

Since 2D barcodes are nothing more than encoding and decoding data, the first thing to decide is what data we would like to encode. Since I actually do not have a business of my own and furthermore use a work issued phone, the data I encode will probably be a URL of my website. There are other interesting encodings, though, which include email address, sms, geographic locations, etc. See zxing’s wiki about BarcodeContents for a better discussion for suggested format, including their primary suggestion of using the MECARD format which is typically a composite of Name, Address, Phone number and Email address.

Once you know what you would like to encode in your 2D barcode, I’m guessing you will need software to help with that. With the assumption that you are not going to be encoding/decoding barcodes with a large frequency, my suggestion is that you use online utilities to help you. Interestingly, it turns out that the google chart api can now generate QR Code online. That is both convenient for repeated generation of QR Code but also in dynamic generation of barcodes. But alas, Jason Delport has created a google app engine application to record your text and generate the QR Code for you by generating the google chart api link for you. At that point, you can either use the supplied URL or simply download the png image. Finally, for performing online decoding of the barcode I have found the zxing online decoder to be the best and least intrusive one available.

The main reason 2D barcodes have not really taken off here in the States is because people have not yet came up with a really good idea to propel it into the mainstream. That, my friends, is going to be up to you and me to accomplish. Or, wait, we could just let google usher it in for us? But seriously I think support for mobile devices to read 2D barcodes is great step forward. Afterwards, I can envision graphic designers coming up with clever barcode prints and ways of intriguing people to scan the code for more details, but then it would be people like us who come up with new categories of data to be encoded in the barcodes for new, innovative ways to use them.

To learn more, try the collection of interesting links provided again by the zxing folks.

I will never say anything is impossible. You can build such a tool and it can have the necessary hooks to allow your associates to customize it to suits your needs. My point is, that work is much harder to pull off than the naive observer might realize. Imagine you are abstracting the details of Suse’s automated installer “AutoYast”. But let’s say the OpenSuse project decides to take a drastic change on how the unattended installer works. Their efforts, no doubt, will be motivated by improving their end user’s experience by presumably making it quicker, simpler and overall a better product. Depending on how drastic the change, it could represent an entirely different philosophical approach to OS installs. As the tool builder, trying to provide a layer of abstraction, you have just stuck yourself into a large endeavour to re-factor those pieces of your application to handle the radical changes being made. It’s a given risk, if that is what you’re providing. My point is, as a customer, just don’t buy that product.

To purchase such a product, you are basically stating that you believe the particular team of developers are going to continue to accurately and intuitively abstract those details for you. Don’t forget you’re still paying a lot of money for this. But this is how management thinks, “I’m going to buy this tool and allow my associates to use one tool and spend their time elsewhere.” It doesn’t happen. Instead, the associates try to shift their energies on learning a new tool, figuring out how to customize it for their needs and probably end up with one FTE dedicated to maintaining it.

Please, don’t waste your time and money. Spend your time collaborating with teammates. Decide upon OS and install standards. Each OS installer provides the ability to perform basic configuration of disk, network, software, etc and then allows for final post-install hook. That hook will then lean upon your team’s efforts. You will end up spending the same amount of work creating your post-install scripts as it takes to merely install and train folks on an “all in one” tool. Big difference of “rolling your own” is you now own the tool set, it already exactly meets your needs, every one knows and understands how it works, updates are easy, knowledge and skill gained is more widely recognized and all the while you haven’t spent more money.

Now for the counter-point: You have to have a good team to pull this off. Team members will require enough experience to demonstrate the proper discernment in building out a quality framework. So what if you maintain a Solaris Jumpstart, RedHat kickstart, Suse autoyast, etc all together? Keep your data and configs centrally managed together. Parallel concepts between each one, maintain like directory hierarchies, write straight forward documentation on using and performing builds. Doesn’t it make sense to be proficient in the OS tool which comes directly from the vendor, at least from a personal development perspective? 

For the demo, I will be leveraging the fail2ban project. It is a python based application which scans typical application logs for security failures and bans IPs from being able to connect again. It also uses the builtin ConfigParser module for reading it’s 30+ config files, which is why I have chosen to use it. For the demo, I have created two scripts:

The first one, configparser2ldap.py is used to process a set of config files and automatically generate LDAP schema as well as LDIF data.

Next, I have my ldapconfig.py module where I extended the ConfigParser module to support making queries to LDAP. I am basically overriding the read() method only and leaving the rest of the module alone. This way the only modifications to the fail2ban application are how it is instantiating the ConfigParser and I won’t have to become a full time fail2ban developer if I want to centralize the configuration data in LDAP.

And that is really the main point of my presentation: The power of centralizing your configuration data and how it can drastically change how you administer your large scale server farm.

Downloads

LDAP + Python Slides.

configparser2ldap.py script to auto-generate LDAP schema and LDIF from ConfigParser compatible config files.

ldapconfig.py python module which inherits the ConfigParser and supports optionally pulling config data from LDAP.

Essential Knowledge

- In case you don’t know, emacs is really just a lisp interpreter.
  Means it’s nearly infinitely extensible.
  P.S. Also, in case you don’t know, Lisp is a programming language. That means practically everything the editor is supporting, from moving the cusor word by word to saving and opening files, is actually being done via code written in Lisp.
- Files being opened and edited are referred to as buffers.
  To say you are “closing a buffer” means you are closing that file and will no longer have it open.
  Similarly, when we say “switch to buffer” means that you are changing the current screen from what you are viewing/editing and moving to the next file.
- In keyboard shortcut notation, (C-x f) means to hold the
<control> key then depress <x>, then releasing and then
pressing <f>.
  In general, a space means you should stop pressing any keys and then move onto the next key sequence.
- Emacs commands are often a two sequence routine.
  Open a file (C-x C-f), save (C-x C-s), quit the editor (C-x C-c).
  Psst. (C-x) is common starting sequence, particularly for the basic operations.
- What does the “M” character stand for in keyboard sequences?
  It is called the “Meta” key and to execute it, you can use the <Alt> key or you can alternatively hit <Esc> followed by the next key. If you use the <Esc> key, you do not depress both keys at the same time, whereas with the <Alt> you do hit both keys at the same time.
  So, to get the command prompt (technically executing execute-extended-command), the sequence is (M-x). With the <Esc> method, it’s hit Escape, then hit “x”, whereas with the <Alt> you would start by holding <Alt> and then depress “x”.
  I used to use the <Esc> key method, but forced myself to use the <Alt> method in order to keep my fingers on the home keys for speed.
- To set a keyboard shortcut, it is called a “binding”.
  The interactive emacs command to do this is called “global-set-key”.
- The equivalent to “man man” of learning shell skills is (C-h ?)
  There you will see various types of help. You can search the building commands, you can query what a keyboard sequence will execute and much more.
- Emacs can do nearly everything, but it is fairly esoteric in it’s approach to being your text editor.
  What this means is that it’s learning curve is pretty steep. If you try to climb that hill initially, you’ll probably give up.
  Try to just learn the essentials and don’t bother with anything else. Learn one thing at a time. Use the menu, use the mouse, use your arrow keys, the pgdn and pgup keys, etc. Try to use the editor just like you would use notepad, but understand that one of the goals of a advanced emacs user is to not need the menu, scroll bars, mouse, etc.
  I once attended a RedHat training course where the instructor called himself a “emacs guy” but watching him actually use the editor appeared to me like he just started using it last week. But hey, that is totally cool, and really, the correct approach for learning emacs.
- Why should I invest in a learning a editor with such a steep learning curve?
  Because it can do nearly anything and can be customized for everything. Without going into a full dissertation, this means that the effort you are investing into this editor will not be put to waste. And moreover, you will not invest efforts just to find that you’ve reached the extent of what the particular editor can do.
- When you use (M-x) to enter a command, if there is a keyboard shortcut, the mini-buffer will tell you what it is.
  Psst: Can also use (C-h b) to show a helping page about the keyboard bindings.
- You can Tab-complete nearly everywhere.
  First place you’ll notice is when opening a file, but you can use within the mini-buffer too. Can be used to complete emacs commands,
  variable names, etc.
- So, you’ve forgotten that command but you remember it’s about the keyboard?
  Search the commands, (C-h a) then just type “key” and ENTER to search for commands with “key” in the name.
- In case you get yourself stuck or just not sure what you just pressed, there are two common ways to quit out.
  (ESC ESC ESC) - Hiting Escape three times gets you out of what you’re doing. This actually works for closing out that split screen help window too.
  The other way to quit out of items is to use (C-g), but there are scenarios where ESC ESC ESC works and C-g doesn’t.
- Customization is done within your ~/.emacs file.
  You are technically executing emacs lisp here. Since you’re reading this, I’ll assume you’re a newbie and as such, just go and copy somebody else’s .emacs file. If you’re still a Emacs user after a year, perhaps you’ll finally feel adventurous enough to dive in there and generate your own customizations.
- Support for different types of files is done via different “modes”.
  Your current “mode” is displayed in the status bar, at the bottom of the editor, in parentheses.
  To change to text mode, for example, execute the command text-mode (M-x text-mode). In general, to enter any mode, it’s going to be the modename followed by “-mode”. Means, for Python, execute python-mode.
  A cheap way to see how many modes Emacs supports is to do a apropos search on “mode”. That is, (C-h a “mode”) to show each command with the word “mode” in it.
- How do I automatically execute something when entering a particular mode?
  Although not an essential piece of knowledge, to execute something when entering a mode to to “create a hook”. By convention, the hooks are stored by the variable matching the mode name and adding “-hook” to the end. So the hook for executing something when I enter “outline” mode is stored in the variable “outline-mode-hook”. I didn’t use that example by accident either. Here is my hook, used in my .emacs file:
(add-hook ‘outline-mode-hook ‘hide-body)
  Even without any Lisp knowledge, that should be fairly intuitive. It will automatically fold the body of each section within my outline document upon opening it.
  Psst: Try sticking a “-*- outline -*-” as the first line of your outline file while keeping the file extension .txt. Without that line, emacs will assume it’s plain text because of the .txt extension, but with that line, you are telling emacs to use outline-mode for this file. It’s great for keeping notes.
  I use it for keeping notes on every single work request ticket I receive at work. Such a file could become beastly in size but I don’t notice since it’s all rolled up when I re-open it.

Essential Habits.

- Keep your buffers open
  Learn to switch between buffers (C-x b TAB)
  I, myself, keep my emacs editor open at all times. I never close it. I keep buffers of my week’s work open for weeks on end. I’ll pretty much only close buffers when it becomes too cluttered when switching between buffers (C-x b TAB).
  Finally, I currently have my EDITOR variable set to “emacsclient” which means commands such as “crontab” connect to that single running emacs editor.
- Tab complete everything.
  Just like you should be doing in the shell, tab-complete commands, filenames, etc within emacs.
- Use the desktop-save feature.
  This tip suppliments the habit of keeping your buffers open. The desktop-save allows you to keep track of all buffers currently open and to reopen them upon restarting emacs. This is useful on laptops where you won’t be able to keep a copy of emacs always running.
- Pay attention to the keyboard shortcuts which are listed in the menu as you navigate to execute them.
  This goes along with the “learn one trick at a time” suggestion. Next time you get tired of executing that command from the menu, instead learn the keyboard shortcut for it.
  Psst: If you are curious what emacs lisp command is being executed in the menu, you can still use the keyboard help command (C-h k). People normally use that help command to tell them what command is being executed for the keyboard sequence but few realize that it works for menu items as well.

Essential Tricks

- (C-x ESC ESC) - Repeat complex sequence
- Want to assign a keyboard shortcut to something which you do alot?
  E.g. By default “goto-line” is not set to a shortcut. Let’s say you want to assign to (C-c g).
         Run (M-x global-set-key). It will prompt you what keyboard sequence to use and what to execute.
         Next run, (C-x ESC ESC) which is for repeat complex routine. At this point, the sequence is listed in emacs-lisp form in the mini-buffer.
         Run (C-x o) to get to the mini-buffer so you can copy that text (C-a C-space C-e M-w) and return to the normal buffer (C-x o).
- If operating in a corporate environment with a jump machine segregating your workstation and the rest of your environment, use TRAMP.
  Tramp allows you edit remote files over various protocols. I recommend using “ssh” and to that point, stick these lines into your .emacs file:

(setq tramp-default-method "ssh")

- Redefine how your backups are made.
  Emacs will automatically make backups of your files as you are editing them. By default it sticks a “~” character at the end of the file.
  Well, this is quite annoying for me since it will also keep file permissions. If I’m creating a script, now I can’t get a unique tab-completion in the shell by typing (./scr TAB) to expand to “./script.sh”. Instead it finds “script.sh~” also. Move your backups to ~/.backups directory. Create that directory and then add these lines to your .emacs file:
(defun my-make-backup-file-name (file-name)
  “Create the non-numeric backup file name for `file-name’.n
   This customized version of this function is useful to keep all backups
   in one place, instead of all over the filesystem.”
  (require ‘dired)
  (message (concat “make-backup: ” file-name))
  (if (file-exists-p “~/.backups”)
      (concat (expand-file-name “~/.backups/”)
          (dired-replace-in-string “/” “|” file-name))
    (concat file-name “~”)))
(setq make-backup-file-name-function ‘my-make-backup-file-name)
- Use emacswiki.org. In fact, the last backup trick was taken from there.
  Just as I recommended above, don’t try to become an expert overnight. Learn the basics well and then pick up one trick at a time. I believe this can be a never ending process as you’re goal should be to become more and more proficient in your editor. So, go out and learn just one new trick and don’t come back until that new trick is ingrained into your finger’s memory.

Essential Memorization List

- (M-x) - Execute emacs command.
- (C-x u) - Undo.
- (M-x revert-buffer) - Oops, I just totally messed this file up and need to revert from the saved copy.
- (C-x o) - Move cursor to other split window pane.
  Nice for looking through the help buffer after it just split your screen.
- (C-x 1) - Keep current buffer and remove other split pane.
  I.e. Get that other window out of my way, please.
- (C-x 0) - Keep the _other_ buffer and remove this one.
  Just like the previous, but you’re keeping the other buffer as the sole window.
  Nice when you’re paging through an apropos section and are done. Use this to close the window without switching back first.
- (C-x C-f) - Open file
- (C-x C-s) - Save file
- (C-x C-c) - Exit

The goal of this short guide was to arm you with the necessary knowledge
in order to get you going with one of the most powerful editors around.
I didn’t necessarilly want to write a whole series of Emacs weblog entries. Instead, I just did a brain dump and I hope it wasn’t too overwhelming. Instead, I’d rather hope it helps
guide you to the island. We’ll play some volleyball, snorkel, hunt peacocks,
etc; you know, island stuff.

I had to solve various issues with my home setup. First the harddrive vulnerability. For that, I bought two 320GB harddrives and setup a /dev/md0 RAID0 mirror. After that, I installed a fresh copy of Fedora Core 9. I got Wordpress back up but honestly my site was still limping along. I left it that way for too long and have finally cleaned things up tonight, culminating the work by upgrading to Wordpress 2.6.2.

Now let’s get back to writing more weblog entries.

What if you got a page and/or ticket for an obscure server’s particular service? The unique problem is that your environment is huge, you’re still relatively new to the company, co-workers are not there to help you and you have never heard of this server. When logging in, you’re hoping that the person has a nice RC script under /etc/init.d/, that you can find the app via a “lsof -i:<port>”, find the application’s home and locate some log files. But what if the application install was not that nice and did not conform to the norms that you are used to?

To either a small or very large degree, you will be reverse engineering this application. If you’re really unlucky, the application who supports it also has no idea about it nor knows anything about Unix-like machines. So, what if there was an application which is polling upon logging into the server, told you, “In case you are looking for the application binX, which typically listens on port XX, it was most likely started last time by issuing the script /path/to/funky/path/binX.sh”. I’m guessing it would freak you out and immediately flood your emotions with confusion, gratitude and curiosity.

So, would such an application be difficult to write?

• Poll any events for read/write/access under key dirs, such as /etc/init.d/, /etc/*conf ? (use the inotify syscall introduced in Linux kernel 2.6.16)
• Track users logging into the system (could correlate later)
• Watch for any new ports being listened on, then record the binary name.
• Reverse engineer this application to automatically collect interesting data on it.
• Intelligently parse an strace (note to self, checkout: http://subterfugue.org/)
• Utilize systemtap for Linux and DTrace for Solaris. pseudo code { observe new socket being opened, so show me the last 10 files opened and executed. correlate application with startup script }

Now, if your data was collected in a easily usable format, you can collect similar data from other machines and start to make broader correlations.

The whole process is really about automating the process of reverse engineering an application. I do that alot. I believe others would like an application which aided or performed the entire reverse engineering for them.

I have been struggling with a production issue at work with a process which has been less than cooperative. You see, I have a java process which gets crazy and starts consuming CPU cycles. When you run a strace against the process the only system call you will see is a sched_yield() call. The java thread is most likely stuck on a spinlock in user space and the process/thread which owns the lock has died or something else, but for my runaway process all it cares about it is checking for it’s lock and yielding execution back to the kernel to schedule another task. Ofcourse, it just gets the CPU again and continues to pound it.

My company pays alot of money for support and we actually have had a case now open for eight months now. The problem is we are unable to gather sufficient data for their level2 and level3 support teams. They would like a javacore to be generated, which can be done by sending a signal 3 to the java process. In addition to a javacore, they recommend sending a signal 11 (SEGV) to the process to prompt the generation of a normal binary core file. Either one would be invaluable for the support team in ascertaining what is going wrong. Unfortunately, it seems that once the process is stuck in this tight, sched_yield() loop any of the signals we send to it are being ignored. In short, that is my problem.

During my Linux Kernel Internals training with RedHat, I had an idea of writing a kernel module to strip the signal handler from the java process so I can finally generate that elusive core file. The kernel module sets up an entry under /proc named stripsignal_pid. If you read the value, it will tell you a quick one-liner about using this interface. To use the module, you write a process ID into that file and that process’s SIGABRT signal handler will be reset to the SIG_DFL. At this point, if you send a SIGABRT signal to the process the result will be writing out of it’s core file.

Download the source along with a helper test program here: stripsignal.tar.gz.

But if all you are interested in is reviewing the short source code, then feel free to browse the stripsignal.c source online.

I tell you what, the best thing I learned from the class was just familiarizing myself with the source code and actually learning some new emacs tricks for navigating large source code projects. Next writeup will be about my experiences with the GNU global tagging system.

When choosing your language, I recommend picking one that has a decent user community, is available on numerous platforms, has had significant time to mature in proving itself and has an extensive modules/library support. Meeting these requirements will leave you using a language that should keep you efficiently producing solutions to your administrative tasks.

First let’s eliminate some languages based on maintainability. Goodbye Haskell, lisp, scheme, Erlang and any other purely functional languages you have used or know of. I’d venture to say that less than 2% of system administrators are comfortable using any one of those languages. And you can obviously not choose a language which only yourself are going to be able to maintain. Aside from staying away from the obscure, the program should be intuitive to read. People can argue on the virtues of their favorite language and why it lends itself to writing maintainable code, but writing maintainable code is truly a skill. You can write obfusticated code in any language. It takes practice and a conscience effort of keeping your code clean and organized well. Here, practice makes perfect, is the key.

Secondly, and in my opinion the most important aspect of the language of choice is staying efficient. Ideally, each program should be succinct and to the point. I no longer use C/C++ regularly, even though that’s the language I started with, because you simply have to write much more code which another language can do in half or less of work. Try looking at one of the ‘P’s of the LAMP stack and see which fits you better and you can see yourself being productive in. That is, evaluate Python, Perl, PHP and Ruby (okay, not a ‘P’ but whatever). Don’t use a language that doesn’t make sense to you. Don’t waste your time.

And finally, time to explain this title and tell a little story where some customer data was delayed during one day’s production incident. One day, we had a production issue where messages were accidentally dequeued from a IBM Webphere MQSeries queue. A tool which was used to grab just one message dequeued all of the messages. To top it all off, the same tool kept seg faulting while trying to requeue the same messages. The solution left to us was to manually parse out each of the discrete messages into separate files. Once in that state, we had another known tool which could upload the messages separately. There were three developers and myself on the phone and we were all racing to the solution. My language of choice was Python and the rest of the developers used the language that they use professionally, Java. So who reached the solution first? Well I wouldn’t be writing this if I hadn’t won, would I? For me, Python makes sense and I can efficiently write code which I like to think other people will be able to understand and update. That is what is most important for your language of choice.

[ As un-entertaining as it is, you can view the Python solution. ]

1. Use meaningful variable names

This point is strictly for the sake of readability. Too often when trying to read somebodies script I’ll actually do various search & replaces of their variables because they used variables like “w”, “w2″, “w3″. It was quick and dirty for the author, but the inheritor of that script would appreciate if you had used more meaningful variable names.

2. Comment your code.

This goes without saying, really…

3. Visually separate any optional settings sections.

Don’t know about you, but sometimes I get lazy and don’t feel like using getopts. Instead, I’ll throw my what would be optional arguments as hard coded variables at the top of my script. I think this is fine, but you’ll want to visually segregate these optional variables from the rest of the script.
I like to use a — dashed line of about 50-70 characters and even put the words “do not modify beyond this point” to further emphasize what you’re encouraged to change and what shouldn’t normally be touched.

4. Use relative pathing for accessing files to the script.

Never assume the user’s cwd is the same as the script and use “./” to run or source another file. I like to set a variable REL_DIR=`dirname $0` and use it to reference the directory where the very script is running from.
E.g. You have a functions script you’d like to source, then with that REL_DIR variable you would “. ${REL_DIR}/<some-file>“.
I’m actually surprised on how often this happens.

5. Always print a usage statement for improper usage and/or when -h option used.

My code excerpt typically looks like:

USAGE="Usage: `basename $0` <my options here>"
if [ -z "$SOME_ARG" ]; then
    print $USAGE 1>&2
    exit 1
fi

6. Conscientiously use STDOUT vs. STDERR in different scenarios.

Not a script faux pau really, but it can help during the development process. Use STDOUT only for informational messages and/or optional debugging info. Then STDERR would only be used for errors. That way, when running the script you can optionally turn off stdout (1>&-) and easily check that nothing was printed to STDERR. When the output is mixed you’ll have a greater chance of missing the error.
One example of this technique in action is when using the tar command. Try leaving out the verbose (’v') option when creating or extracting your archive, then you can easily see when you might have had a permissions issue or something else related.

7. Keep all required variables defined in the script.

Define the required variables at the top of the script. Even mention that they are REQUIRED. A good example of this is scripts that use Sybase’s isql utility. Anytime I run isql, I like to set something like:

# required variables for isql
SYBASE=/some/path/to/sybase
LD_LIBRARY_PATH=$SYBASE/lib

What you want to avoid is a situation where the script works because you’ve got the required variable set in your env, but only because it’s set in one of your dot files.

8. Cron’ed Scripts.

Two common principles I like to emphasize here:
1. Keep all required variable/env settings in the script! cron does NOT source your dot files.
2. Redirect stdout, but leave stderr unmanaged. This is a cheap technique, but whenever I don’t have time to test for all possible errors I simply setup my .forward file and let cron email me the output produced from the cron script. Though, to be complete, you should really manage your stderr in other fashions.

9. Keep your exit/return codes categorized.

Not always important for small scripts, but a good practice.
For any sort of error checking your script might perform, use a unique error code for each situation that you decide to exit the shell script. That will make invocations of your script more manageable.

10. Avoid “Magic” Numbers

Anywhere you are comparing a value to some, seemingly, arbitrary number, go ahead and set that value to a meaningful variable name. Then your comparison reads alot better.
Using “$CURRENT_VALUE -gt $THRESHOLD” is much better than finding “$CURRENT_VALUE -gt 83” buried in some script and not having any clue what the number 83 signifies aside from the surrounding code.

11. Use unique temporary files.

Never do this: /path/to/some/command -option > command.out.
You are assuming that you are sitting in a directory where you have permissions to create a temporary file and secondly that no one will ever be running the same script at the same time you are.
Some shells make creating temporary files easy with commands such as mktemp. I typically employ a convention where I define my temporary file space as “TEMP=/tmp/.myshellname$$_“. Then lets say I need a temp file to capture the output from ps. I might redirect it to ${TEMP}raw_ps.
And finally, at the end of the script, or defined in a shell function, you can cleanup each temporary file with one line: rm -f ${TEMP}*.

In general, well written code/scripts should read well and be organized well. Every principle discussed above has one purpose: maintainability.