jonEbird

I sit in the Atlanta Airport reminiscing over the events of PyWorks ‘08. This was the first year for PyWorks but MTA combined the conference with PHP Architect and I believe everyone was happy with the combination. At a minimum, people had engaging conversations between the groups and a significant number of them cross-attended the sessions. I attended two PHP sessions and one neutral session and then the rest Python. Some people were a bit disappointed in the lack of Python attendees and it is true that we didn’t make up a large part of the total 148 attendees of the conference. But with the quality of talks staying superbly high, not having a full room wasn’t a bad thing.

The quality of talks were all superb, indeed. Probably over half of the presenters are either principle developers on high profile projects or they have written a book or own their own consulting company. On day zero, where there were 3hr long tutorial sessions, I spend the morning in Mark Ramm’s TurboGears but then I switched over to the PHP side in the afternoon to catch Scott MacVicar and Helgi Þormar Þorbjörnsson’s Caching for Cash.

At the start of day one, the first day of the normal sessions, I think everyone was expecting a lot more people. There were, in fact, more people but not as many as I was expecting, but again that’s perfectly okay. This day was a full one, starting off with the keynote by Kevin Dangoor about Growing your Community. After a break I then attended Decorators are Fun by Matt Wilson and learned that he is not that far away from me in Cleveland. Next I attended another Mark Ramm talk about WSGI where he was explaining how easy it was to build a web framework. It was given a bit “tongue in check” since he is the primary maintainer of TurboGears. Following that, I attended a middle track session about Distributed version control with GIT by Travis Swicegood. Travis had just finished writing a book about using GIT called Pragmatic Version Control Using Git and not surprisingly gave a authoritation explanation of using GIT. Following lunch, I attending another PHP track presentation but it could have been in the neutral middle track. The talk was Map, Filter, Reduce In the Small and in the Cloud by Sebastian Bergmann where he explained the popular functional programming techniques popularized by Google for computing large quantities of data. Sebastian gave me another reason to checkout Hadoop and in fact I’m now thinking of another Python Magazine article about using hadoop with Jython. For the last session of the day I decided to attend Michael Foord’s talk about IronPython. I didn’t think I’d ever checkout IronPython on my own, so I thought I’d get a crash course from Michael who also just finished work on his book IronPython in Action.

Still not done with day one. After all of the normal presentation’s concluded, we had happy hour while gearing up for the Pecha Kucha competition sessions. Pecha Kucha is where you provide 20 slides and set them to auto switch every 20 seconds making your session a little over six minutes. Apparently people have found that you can get the same quality bits of information in that format as compared to a full hour session. At least that is what the Japanese have concluded. As for PHP/PyWorks, we mostly had fun with the sessions. There were talks about web security, general ranting, LOLCode, and many others which I’m having a problem remembering. At the end, the LOLCode talk took the prize of the Xbox 360 gaming system by our judges and if you’d really like to see what was going on, you may be able to watch streamed video captured by Travis Swicegood’s iPhone. Before I went to bed, I rehearsed my presentation one more time.

By the time day two started, it felt like I had been there a full week and yet we still had a full day of presentations again. I started the morning in Chris Perkins’s talk about the Sphinx Documentation System. We all understand the importance of documentation and it’s not always fun, but again I thought investing 45min catching up on some of the Python “best practices” for documentation would be well worth the time. Afterwards, I stayed in the same room for Jacob Taylor’s talk about Exploring Artificial Intelligence with Python. Jacob didn’t get around to showing any Python code but he had good attendance for being a founder of SugarCRM. Next, the highlight of the conference, my presentation about LDAP and Python. The number of attendees for my presentation were average for the Python sessions and by this point I felt like I knew everyone which removed any pressure or nervousness. We’ll see how interested people were by seeing who downloads my configparser.py and/or ldapconfig.py scripts. After lunch, I attended Kevin Dangoor’s Paver talk where he explained the motivations for Paver and showed numerous examples of what pain points it solves. Finally, the last session I attended at PyWorks was Jonathan LaCour’s talk about Elixir, the Python module which makes introduction into SQLAlchemy an easy one. Elixir helps kick start your DB code by simplifying SQLAlchemy by making a lot of sane choices for you as well as providing other conveniences. Jonathan had to work hard to get all of his content into his hour, mostly because he gave a decent overview of SQLAlchemy and then his Elixir module.

As with the previous day, this day concluded with another happy hour while waiting for our closing keynote. The closing keynote was given by Jay Pipes about “living in the gray areas” and not sticking to extreme black and white of our technologies. He praised the joint efforts being made by the PHP and Python folks and criticized people who are too biased to learn from the other communities. Jay is working on Drizzle, while working for Sun, where they are challanging all of the preconceived notions being made by the MySQL community. Drizzle is basically a fork of MySQL and their goals are to provide a much more streamlined version of a database. Jay explained that forks are good (as well as “sporks”) because it keeps people on their toes and keeps the level of competition up. Finally, Jay’s last point was that we need to spend more time listening to other people and less time preaching our biased opinions.

I overheard PHP and Python people resonating Jay’s message after the keynote. I’m glad to have participated in such a successful conference where I truely believe boundries were crossed. With as much time that I spend with the PHP folks, I was repeatedly asked, “So, you coming over to the PHP side?” I think the last time I was asked that was in the hotel pool where again I was playing the role of the “token Python guy” amongst the PHP folks. To be honest, those PHP folks know how to have fun, and if my criteria for choosing a programming language was the amount of fun the community had I would be doing PHP development. I definately want attend next year’s PyWorks and PHP conference and I have an entire year to come up with my presentation proposals.

In anticipation of heading down to PyWorks 2008, I have been thinking about creating a business card for the sake of keeping in contact with people I meet. One of my main goals, while attending and speaking at PyWorks, is to network with people and mark 2008 as the year which I start participating and contributing within the OSS community. While exercising my creativity in designing a nice business card, I have also been reading about Google’s android mobile platform, and I came across a interesting intersection between the two when I saw a demonstration video.

A Google developer, working on the zxing project (pronounced “zebra crossing”), has printed a 2D barcode encoding of his personal information on the back of his business card. With the builtin camera, on his android phone, he can scan in a barcode and immediate use the encoded data. It is an impressive demonstration of integrating technology with our mobile devices. Check out the video which has inspired me to not only do the same but also write this small informational note about 2D barcodes.

If you didn’t catch it, the format of the 2D barcode on his business card is QR Code. Among the other 2D barcode formats, QR Code barcodes are most popular in Japan where it was invented by Denso-Wave. The popularity of QR Codes in Japan has grown to the level of being supported by nearly every mobile device there and that also means finding QR Codes available on a increasing amount of printed media from fliers to magazines and coupons.

There are other competing 2D barcode formats that I could choose from but after doing researching and not seeing any distinctive advantages, I have concluded to follow suit with the google developer and hope that the android phone’s application and popularity will help propel QR Code’s popularity over the other 2D barcode formats.

Since 2D barcodes are nothing more than encoding and decoding data, the first thing to decide is what data we would like to encode. Since I actually do not have a business of my own and furthermore use a work issued phone, the data I encode will probably be a URL of my website. There are other interesting encodings, though, which include email address, sms, geographic locations, etc. See zxing’s wiki about BarcodeContents for a better discussion for suggested format, including their primary suggestion of using the MECARD format which is typically a composite of Name, Address, Phone number and Email address.

Once you know what you would like to encode in your 2D barcode, I’m guessing you will need software to help with that. With the assumption that you are not going to be encoding/decoding barcodes with a large frequency, my suggestion is that you use online utilities to help you. Interestingly, it turns out that the google chart api can now generate QR Code online. That is both convenient for repeated generation of QR Code but also in dynamic generation of barcodes. But alas, Jason Delport has created a google app engine application to record your text and generate the QR Code for you by generating the google chart api link for you. At that point, you can either use the supplied URL or simply download the png image. Finally, for performing online decoding of the barcode I have found the zxing online decoder to be the best and least intrusive one available.

The main reason 2D barcodes have not really taken off here in the States is because people have not yet came up with a really good idea to propel it into the mainstream. That, my friends, is going to be up to you and me to accomplish. Or, wait, we could just let google usher it in for us? But seriously I think support for mobile devices to read 2D barcodes is great step forward. Afterwards, I can envision graphic designers coming up with clever barcode prints and ways of intriguing people to scan the code for more details, but then it would be people like us who come up with new categories of data to be encoded in the barcodes for new, innovative ways to use them.

To learn more, try the collection of interesting links provided again by the zxing folks.

The challenge to build a tool which manages multiple vendors and platforms by way of piggy backing off their technology is a losing battle. Be it provisioning, patching, monitoring, etc it doesn’t matter. To choose such a tool, you end up paying big bucks for other people to constantly watch and react to what various vendors are doing. Combine that piece of realization with the fact that a tool will almost never perfectly suit the unique requirements of your business and you’d be in denial to not realize that it sucks. Beyond the shear money of the endeavour you are also wasting time of your associates which will probably not get recouped.

I will never say anything is impossible. You can build such a tool and it can have the necessary hooks to allow your associates to customize it to suits your needs. My point is, that work is much harder to pull off than the naive observer might realize. Imagine you are abstracting the details of Suse’s automated installer “AutoYast”. But let’s say the OpenSuse project decides to take a drastic change on how the unattended installer works. Their efforts, no doubt, will be motivated by improving their end user’s experience by presumably making it quicker, simpler and overall a better product. Depending on how drastic the change, it could represent an entirely different philosophical approach to OS installs. As the tool builder, trying to provide a layer of abstraction, you have just stuck yourself into a large endeavour to re-factor those pieces of your application to handle the radical changes being made. It’s a given risk, if that is what you’re providing. My point is, as a customer, just don’t buy that product.

To purchase such a product, you are basically stating that you believe the particular team of developers are going to continue to accurately and intuitively abstract those details for you. Don’t forget you’re still paying a lot of money for this. But this is how management thinks, “I’m going to buy this tool and allow my associates to use one tool and spend their time elsewhere.” It doesn’t happen. Instead, the associates try to shift their energies on learning a new tool, figuring out how to customize it for their needs and probably end up with one FTE dedicated to maintaining it.

Please, don’t waste your time and money. Spend your time collaborating with teammates. Decide upon OS and install standards. Each OS installer provides the ability to perform basic configuration of disk, network, software, etc and then allows for final post-install hook. That hook will then lean upon your team’s efforts. You will end up spending the same amount of work creating your post-install scripts as it takes to merely install and train folks on an “all in one” tool. Big difference of “rolling your own” is you now own the tool set, it already exactly meets your needs, every one knows and understands how it works, updates are easy, knowledge and skill gained is more widely recognized and all the while you haven’t spent more money.

Now for the counter-point: You have to have a good team to pull this off. Team members will require enough experience to demonstrate the proper discernment in building out a quality framework. So what if you maintain a Solaris Jumpstart, RedHat kickstart, Suse autoyast, etc all together? Keep your data and configs centrally managed together. Parallel concepts between each one, maintain like directory hierarchies, write straight forward documentation on using and performing builds. Doesn’t it make sense to be proficient in the OS tool which comes directly from the vendor, at least from a personal development perspective? 

I am on a island at work and there is no one else to converse with other than Wilson. The island is called “Isle de Emacs” and Wilson is, I guess, psychoanalyze-pinhead. But occasionally I do get a visitor to the island and the conversation is more or less the same stuff. Okay, enough with the analogy, this short tutorial is about getting you acquainted with using the Emacs editor. The repeated conversation I was referring to was an introduction to Emacs and trying to educate my co-worker on both the essential pieces of knowledge and a bit of Emacs philosophy to help guide their learning process.

Essential Knowledge

- In case you don’t know, emacs is really just a lisp interpreter.
  Means it’s nearly infinitely extensible.
  P.S. Also, in case you don’t know, Lisp is a programming language. That means practically everything the editor is supporting, from moving the cusor word by word to saving and opening files, is actually being done via code written in Lisp.
- Files being opened and edited are referred to as buffers.
  To say you are “closing a buffer” means you are closing that file and will no longer have it open.
  Similarly, when we say “switch to buffer” means that you are changing the current screen from what you are viewing/editing and moving to the next file.
- In keyboard shortcut notation, (C-x f) means to hold the
<control> key then depress <x>, then releasing and then
pressing <f>.
  In general, a space means you should stop pressing any keys and then move onto the next key sequence.
- Emacs commands are often a two sequence routine.
  Open a file (C-x C-f), save (C-x C-s), quit the editor (C-x C-c).
  Psst. (C-x) is common starting sequence, particularly for the basic operations.
- What does the “M” character stand for in keyboard sequences?
  It is called the “Meta” key and to execute it, you can use the <Alt> key or you can alternatively hit <Esc> followed by the next key. If you use the <Esc> key, you do not depress both keys at the same time, whereas with the <Alt> you do hit both keys at the same time.
  So, to get the command prompt (technically executing execute-extended-command), the sequence is (M-x). With the <Esc> method, it’s hit Escape, then hit “x”, whereas with the <Alt> you would start by holding <Alt> and then depress “x”.
  I used to use the <Esc> key method, but forced myself to use the <Alt> method in order to keep my fingers on the home keys for speed.
- To set a keyboard shortcut, it is called a “binding”.
  The interactive emacs command to do this is called “global-set-key”.
- The equivalent to “man man” of learning shell skills is (C-h ?)
  There you will see various types of help. You can search the building commands, you can query what a keyboard sequence will execute and much more.
- Emacs can do nearly everything, but it is fairly esoteric in it’s approach to being your text editor.
  What this means is that it’s learning curve is pretty steep. If you try to climb that hill initially, you’ll probably give up.
  Try to just learn the essentials and don’t bother with anything else. Learn one thing at a time. Use the menu, use the mouse, use your arrow keys, the pgdn and pgup keys, etc. Try to use the editor just like you would use notepad, but understand that one of the goals of a advanced emacs user is to not need the menu, scroll bars, mouse, etc.
  I once attended a RedHat training course where the instructor called himself a “emacs guy” but watching him actually use the editor appeared to me like he just started using it last week. But hey, that is totally cool, and really, the correct approach for learning emacs.
- Why should I invest in a learning a editor with such a steep learning curve?
  Because it can do nearly anything and can be customized for everything. Without going into a full dissertation, this means that the effort you are investing into this editor will not be put to waste. And moreover, you will not invest efforts just to find that you’ve reached the extent of what the particular editor can do.
- When you use (M-x) to enter a command, if there is a keyboard shortcut, the mini-buffer will tell you what it is.
  Psst: Can also use (C-h b) to show a helping page about the keyboard bindings.
- You can Tab-complete nearly everywhere.
  First place you’ll notice is when opening a file, but you can use within the mini-buffer too. Can be used to complete emacs commands,
  variable names, etc.
- So, you’ve forgotten that command but you remember it’s about the keyboard?
  Search the commands, (C-h a) then just type “key” and ENTER to search for commands with “key” in the name.
- In case you get yourself stuck or just not sure what you just pressed, there are two common ways to quit out.
  (ESC ESC ESC) - Hiting Escape three times gets you out of what you’re doing. This actually works for closing out that split screen help window too.
  The other way to quit out of items is to use (C-g), but there are scenarios where ESC ESC ESC works and C-g doesn’t.
- Customization is done within your ~/.emacs file.
  You are technically executing emacs lisp here. Since you’re reading this, I’ll assume you’re a newbie and as such, just go and copy somebody else’s .emacs file. If you’re still a Emacs user after a year, perhaps you’ll finally feel adventurous enough to dive in there and generate your own customizations.
- Support for different types of files is done via different “modes”.
  Your current “mode” is displayed in the status bar, at the bottom of the editor, in parentheses.
  To change to text mode, for example, execute the command text-mode (M-x text-mode). In general, to enter any mode, it’s going to be the modename followed by “-mode”. Means, for Python, execute python-mode.
  A cheap way to see how many modes Emacs supports is to do a apropos search on “mode”. That is, (C-h a “mode”) to show each command with the word “mode” in it.
- How do I automatically execute something when entering a particular mode?
  Although not an essential piece of knowledge, to execute something when entering a mode to to “create a hook”. By convention, the hooks are stored by the variable matching the mode name and adding “-hook” to the end. So the hook for executing something when I enter “outline” mode is stored in the variable “outline-mode-hook”. I didn’t use that example by accident either. Here is my hook, used in my .emacs file:
(add-hook ‘outline-mode-hook ‘hide-body)
  Even without any Lisp knowledge, that should be fairly intuitive. It will automatically fold the body of each section within my outline document upon opening it.
  Psst: Try sticking a “-*- outline -*-” as the first line of your outline file while keeping the file extension .txt. Without that line, emacs will assume it’s plain text because of the .txt extension, but with that line, you are telling emacs to use outline-mode for this file. It’s great for keeping notes.
  I use it for keeping notes on every single work request ticket I receive at work. Such a file could become beastly in size but I don’t notice since it’s all rolled up when I re-open it.

Essential Habits.

- Keep your buffers open
  Learn to switch between buffers (C-x b TAB)
  I, myself, keep my emacs editor open at all times. I never close it. I keep buffers of my week’s work open for weeks on end. I’ll pretty much only close buffers when it becomes too cluttered when switching between buffers (C-x b TAB).
  Finally, I currently have my EDITOR variable set to “emacsclient” which means commands such as “crontab” connect to that single running emacs editor.
- Tab complete everything.
  Just like you should be doing in the shell, tab-complete commands, filenames, etc within emacs.
- Use the desktop-save feature.
  This tip suppliments the habit of keeping your buffers open. The desktop-save allows you to keep track of all buffers currently open and to reopen them upon restarting emacs. This is useful on laptops where you won’t be able to keep a copy of emacs always running.
- Pay attention to the keyboard shortcuts which are listed in the menu as you navigate to execute them.
  This goes along with the “learn one trick at a time” suggestion. Next time you get tired of executing that command from the menu, instead learn the keyboard shortcut for it.
  Psst: If you are curious what emacs lisp command is being executed in the menu, you can still use the keyboard help command (C-h k). People normally use that help command to tell them what command is being executed for the keyboard sequence but few realize that it works for menu items as well.

Essential Tricks

- (C-x ESC ESC) - Repeat complex sequence
- Want to assign a keyboard shortcut to something which you do alot?
  E.g. By default “goto-line” is not set to a shortcut. Let’s say you want to assign to (C-c g).
         Run (M-x global-set-key). It will prompt you what keyboard sequence to use and what to execute.
         Next run, (C-x ESC ESC) which is for repeat complex routine. At this point, the sequence is listed in emacs-lisp form in the mini-buffer.
         Run (C-x o) to get to the mini-buffer so you can copy that text (C-a C-space C-e M-w) and return to the normal buffer (C-x o).
- If operating in a corporate environment with a jump machine segregating your workstation and the rest of your environment, use TRAMP.
  Tramp allows you edit remote files over various protocols. I recommend using “ssh” and to that point, stick these lines into your .emacs file:

(setq tramp-default-method "ssh")

- Redefine how your backups are made.
  Emacs will automatically make backups of your files as you are editing them. By default it sticks a “~” character at the end of the file.
  Well, this is quite annoying for me since it will also keep file permissions. If I’m creating a script, now I can’t get a unique tab-completion in the shell by typing (./scr TAB) to expand to “./script.sh”. Instead it finds “script.sh~” also. Move your backups to ~/.backups directory. Create that directory and then add these lines to your .emacs file:
(defun my-make-backup-file-name (file-name)
  “Create the non-numeric backup file name for `file-name’.n
   This customized version of this function is useful to keep all backups
   in one place, instead of all over the filesystem.”
  (require ‘dired)
  (message (concat “make-backup: ” file-name))
  (if (file-exists-p “~/.backups”)
      (concat (expand-file-name “~/.backups/”)
          (dired-replace-in-string “/” “|” file-name))
    (concat file-name “~”)))
(setq make-backup-file-name-function ‘my-make-backup-file-name)
- Use emacswiki.org. In fact, the last backup trick was taken from there.
  Just as I recommended above, don’t try to become an expert overnight. Learn the basics well and then pick up one trick at a time. I believe this can be a never ending process as you’re goal should be to become more and more proficient in your editor. So, go out and learn just one new trick and don’t come back until that new trick is ingrained into your finger’s memory.

Essential Memorization List

- (M-x) - Execute emacs command.
- (C-x u) - Undo.
- (M-x revert-buffer) - Oops, I just totally messed this file up and need to revert from the saved copy.
- (C-x o) - Move cursor to other split window pane.
  Nice for looking through the help buffer after it just split your screen.
- (C-x 1) - Keep current buffer and remove other split pane.
  I.e. Get that other window out of my way, please.
- (C-x 0) - Keep the _other_ buffer and remove this one.
  Just like the previous, but you’re keeping the other buffer as the sole window.
  Nice when you’re paging through an apropos section and are done. Use this to close the window without switching back first.
- (C-x C-f) - Open file
- (C-x C-s) - Save file
- (C-x C-c) - Exit

The goal of this short guide was to arm you with the necessary knowledge
in order to get you going with one of the most powerful editors around.
I didn’t necessarilly want to write a whole series of Emacs weblog entries. Instead, I just did a brain dump and I hope it wasn’t too overwhelming. Instead, I’d rather hope it helps
guide you to the island. We’ll play some volleyball, snorkel, hunt peacocks,
etc; you know, island stuff.

Some time ago I lost my harddrive. At that point, I was making infrequent backups to the laptop. Not very admin-like for someone who’s profession is a Systems Administrator. It could have been much worse, but still a pain none the less.

I had to solve various issues with my home setup. First the harddrive vulnerability. For that, I bought two 320GB harddrives and setup a /dev/md0 RAID0 mirror. After that, I installed a fresh copy of Fedora Core 9. I got Wordpress back up but honestly my site was still limping along. I left it that way for too long and have finally cleaned things up tonight, culminating the work by upgrading to Wordpress 2.6.2.

Now let’s get back to writing more weblog entries.

Today I turn 28 and I just received some well timed news from my place of work. Yesterday I was promoted to a Senior level administrator after three years of service.

I started with the company in June of 2000 while still attending Ohio State University and I worked as an intern for over three years when I was finally hired full time. Although my company is not huge, it is profitable and has been in the computer industry for over 30 years. I always thought I would have to move around to truly gain broad experience for my resume, but instead my company is remarkable in how much it changes. As a result, I get a stable working position while helping myself to some welcomed experience.

As an intern, I was doing development work in C/C++, sql & shell scripting primarily. I also had a side job at the University helping people out in the lab which helped fuel my continued interest in administration. When I decided to shift to an administration role, I spend quite a bit of time just reading books at the bookstore, buying some, checking some out from the library, but mostly just reading them while enjoying some coffee in the cafe. The career development goal was simple: learn as much as you can in the field of administration and the operating systems you are supporting. Now that I’ve seemingly achieved that goal, as seen by my promotion, what should my development goal be now?

What does a lead administrator do that a senior administrator just isn’t up for? I am sure I can read the official job description from my HR department, but the last time I looked it was severely out of date. The road from entry level or associate to Senior is one that is primarily technical, at least in my field, where you are merely demonstrating yourself as proficient in handling challenges, communicate well, work well with others and make your deadlines. The next level, I suspect, will require myself venturing out in the areas where nerds are not comfortable going.

A lead administrator is involved in budget discussions, aids in design decisions along with the architects, understands the business justifications and ramifications to technical decisions. Honestly, not all of these aspects are that appealing to me. Instead, when I have free time of my own to explore interesting, work related topics, it almost always involving some sort of programming. If I’m not creating some mini utility that I’ve really wanted to have, I’m reading some book or blog about programming, checking out the latest language war threads or even evaluating a popular language I’ve yet to mess around with.

My ideal job is one where I do not lose my root access, get to play with really cool technology, am looked to for technical and architectural decisions, and get to frequently write supporting programs to further our administration group, monitoring teams, database administrators and even the development staff. My latest development utility developed at work was to abstract all of our add-on program’s init scripts into a new schema in LDAP to be centralized administrated. The problem is I do not know of a official job that matches my 50/50 mix of administration and development.

In conclusion, I’m going to take it easy while celebrating my b-day and just revel in my promotion for a while. As for the coming years, I like to flirt with the idea of launching a new startup company. I can develop my own software and still handle the administration of our machines. Now if I could just grow the balls to do so…

Time will tell what this site will become… But I promise you this much: I won’t write crap just to fill in content.

Instead, this will merely be a news heading forum for all things being and to be hosted here.