Some links of interest

• Nov. 4, 2008 Poll Closing Times & Key Races
• FiveThirtyEight
• Presidential Interactive Map
• Paths to 270

Continuing a meme I discovered here (though starting here and also seen here), the ordered list of programming languages I've learned:

1. BASIC 1981
   1. the original line BASIC for the Commodore VIC-20
   2. GW-BASIC / QuickBASIC 1991
2. C 1992
3. C++ 1993
4. Pascal 1994
5. Scheme 1994
6. Assembly (for Motorola 68K) 1994
7. Lisp 1995 - specifically, the AutoLisp dialect from AutoDesk for AutoCAD
8. COBOL 1997
9. *NIX Shell 1997
   1. Korn (ksh)
   2. csh 2000
   3. bash 2006
10. Perl 1999 - makes me queasy to admit
11. Python 2000 - my current language of choice
12. Javascript 2001
13. FORTRAN 2002
14. C# 2003
15. Visual Basic 2003
16. AppleScript 2005
17. Objective-C 2005
18. Java 2006

In no way am I making any claims to the continued competency in the majority of these languages. In fact, I've been very successful at expunging most from memory.

Additionally, I'm listing the procedural, expressive, or non-computational languages I've learned:

1. MS DOS Batch
2. Makefile
3. SQL
4. DB2
5. HTML
6. XML
7. XSLT

And just for kicks, a list of operating systems:

1. DOS
   1. MS DOS - versions 1, 4-6
   2. PC DOS
   3. DR-DOS
2. MS Windows
   1. 3, 3.1x
   2. 9x
   3. NT 4 | 2000 | XP | 2K3
3. VAX VMS
4. *NIX
   1. BSD
   2. HP-UX
   3. Solaris
   4. AIX
   5. Linux - Red Hat, Debian, Ubuntu
5. IBM System/370
6. OS/2 | Warp
7. Mac OS 9
8. Mac OS X (10.x)

Since DjangoCon, there seems to be a fair amount of discussion centered around the comparison of, drawbacks of, and paths to improving Django's templating engine.

This write is not intended to be a discussion on the principals or philosophies of Django's approach to templates verses the alternatives. I will, however, that I fully understand and appreciate the desire for the separation of concerns (i.e., developer vs. designer), although working mostly as a team of one, I personally find this approach to be a bit restricting. I don't know that "jumping ship" to Jinja is necessarily the best option for me. Instead, finding ways to improve Django templates and contributing back to the Django community feels right to me.

Having said that, I must admit that I find writing template extensions to be the single most tedious and cumbersome aspects of using Django. It feels like I am constantly repeating myself - DRY alert! - by writing new template tags: start by writing the micro-parser for the tag syntax, follow that by creating the wrapper Node class extension, and only then getting to the point where I can actually write the business or domain-specific logic that is needed. I am forever looking up references and consulting past work because there seems to be nothing intuitive or natural about the process. This is major put-off, and instead I would find myself putting the work of populating my context data into the individual views - which also didn't seem very DRY, nor very reusable.

I've seen a few suggestions for patches to help minimize the effort required to extend tags template. Until one of these patches - past, present, or future - finds it's way into trunk - or a release, in many developer's situation - I thought that I would share a solution that I have been using for awhile now that can be used in the interim.

Rather than list all the source code here, I've zipped up a simple project that exposes a Library extension and a sample application that takes advantage of it using various examples. The download can be found here.

The basic idea is this: provide an means of accessing something (a queryset, any arbitrary Python object) and injecting it into the template context as a named variable via a simple function that has been automagically been made into a new template tag by way of smart decorator. For flexibility, passing arguments into the function is an optional feature. I like to think of these new template-tagged functions as tag getters.

A somewhat contrived example might help to illustrate. First, the template code itself: I want to be able to retrieve a list of active users, possibly filtered. Currently, my most direct option would be to perform the query within the view and pass along the resulting queryset to the template through the Context instance. It's easy to see that this can become quite irksome: perhaps I have multiple views where this is necessary, or the variations on the filter are specific enough to make it overly complex to all the possible results to the template in manner where it can be consumed in a sensible fashion

Consider the following example that filters active users by applying some regular expressions (I warned that this would be contrived):

1
2
3
4
5
6
7

{% get_active_users as active_users using "^L" "(x|z)" %}
<p>Active users: {{ active_users.count }}</p>
<ul>
    {% for active_user in active_users %}
    <li>{{ active_user.get_full_name }}</li>
    {% endfor %}
</ul>

Get me the active users whose first or last name begins with L or contains either x or z (note: I could have easily done this by using a single regex argument, but I wanted to show multiple arguments being passed). The result, in this case a queryset, is assigned to the variable active_users. using and the subsequent arguments are optional in this case, as we'll see shortly.

Creating the get_active_users template tag is very simple and pretty much only requires thought to be given to the actual problem of user retrieval, allowing the developer to bypass most of the mechanics and build up of necessary template extension foo:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

import operator
from django.db.models import Q
from django.contrib.auth.models import User
from template_utils import GetterLibrary

register = GetterLibrary()

@register.getter_tag
def get_active_users(*regex_filters):
    qs = User.objects.filter(is_active=True)
    if regex_filters:
        filter_by = [
            Q(first_name__regex=regex_filter) | Q(last_name__regex=regex_filter)
            for regex_filter in regex_filters
        ]
        qs = qs.filter(reduce(operator.or_, filter_by))

    return qs

So, we have a shortcut to producing a new template involving really only three steps, 2 of which we would have had to do regardles:

1. template_utils.GetterLibrary is a simple class derived from django.template.Library. I have point out here that my naming skills are lame at best, so please forbear from snickering - I welcome suggestions for a better choice of labels. GetterLibrary does everything we know and love from the base Library class, plus:

2. The getter_tag decorator method of the GetterLibrary instance, which provides the bit of function-to-tag automation, and:

3. Our "worker" function, whose sole job is to return any result we wish to assign to our new context variable.

There are several more examples to be found in the test project, so please download it it and poke about through the code and demo app.

Even though Leopard ships with a fairly recent version of Python (2.5.1), there are several reasons one would want to use an alternative installation. For instance, Python 2.3.5 shipped with Tiger and was growing pretty long in the tooth by the time Leopard arrived; 2.3 was the default installation on Panther.

I've tried a few approaches to installing newer versions on the Mac: several years ago, I played around with Fink, then switched to MacPorts (formerly DarwinPorts). While either of these package managers make it very easy to install various software pacakages on the Mac, they each presented certain problems or limitations with respect to Python and the other web development packages I need.

My current preferred solution is to go with the MacPython distribution found on the Python website. There is also a great SIG (http://mail.python.org/mailman/listinfo/pythonmac-sig) for getting help.

From http://www.python.org/download/mac/:

Python comes pre-installed on Mac OS X, but due to Apple's release cycle, it's often one or even two years old. The overwhelming recommendation of the "MacPython" community is to upgrade your Python by downloading and installing a newer version.

Listed below are the steps to setting up some of the most common Python packages used to for web development, particularly with respect to Django.

Getting started

Basic assumptions

• Mac OS X Leopard (10.5) - as of the date of this posting, I am on 10.5.3

• Subversion - you can find an installer at http://subversion.tigris.org/

• I use MySQL, so instructions for installing it and the necessary Python bindings (MySQLdb) are shown. If anyone has detailed instructions for Postgres installation, please send them to me at python [at] dakrauth.com

• An installation directory to use for these steps. For instance, here's how I began (from a Terminal window, bash shell):

  1 2 3

  mkdir -p ~/python/install mkdir ~/python/site-packages cd ~/python/install

  I use ~/python/site-packages as an easy way of referencing packages that will change frequently or for creating a pseudo-virtual environment. More on this below.

Installations

Python 2.5.2

• MacPython installation:

  1 2 3 4

  curl -O http://www.python.org/ftp/python/2.5.2/python-2.5.2-macosx.dmg open python-2.5.2-macosx.dmg open /Volumes/Universal\ MacPython\ 2.5.2/MacPython.mpkg hdiutil unmount /Volumes/Universal\ MacPython\ 2.5.2/

  This installation will create the site-packages directory at:

  /Library/Frameworks/Python.framework/Versions/Current/lib/python2.5/site-packages
  

  Additionally, your .profile file will be updated to place the above path at the front of you $PATH environment variable. I usually remove this change, since the only thing I like to have my .profile file a source line to my .bashrc file. However, I ensure that /usr/local/bin is place ahead /usr/bin in my $PATH string.

• Now I set up the ties to the aforementioned local site-packages directory:

  1 2 3

  pushd /Library/Frameworks/Python.framework/Versions/Current/lib/python2.5/site-packages echo /Users/your-username/python/site-packages > my.pth popd

MySQL

• Lines 4, 5, and 6 below are a quick hack around an odd bug in the installer.

  1 2 3 4 5 6

  curl -O http://mirror.services.wisc.edu/mysql/Downloads/MySQL-5.0/mysql-5.0.51b-osx10.5-x86.dmg open mysql-5.0.51b-osx10.5-x86.dmg open /Volumes/mysql-5.0.51b-osx10.5-x86/mysql-5.0.51b-osx10.5-x86.pkg/ pushd /usr/local/mysql/lib sudo ln -s /usr/local/mysql/lib mysql popd

• Add the preference pane to your System Preferences and start the server from the preference pane (I also prefer to select Automatically Start MySQL Server on Startup):

  1 2	open /Volumes/mysql-5.0.51b-osx10.5-x86/MySQL.prefPane/ hdiutil unmount /Volumes/mysql-5.0.51b-osx10.5-x86/

• To setup a sample database to use with a Django project, here are the commands I issue from within the mysql shell:

  1 2 3 4 5 6

  CREATE DATABASE spam_db CHARACTER SET utf8 COLLATE utf8_general_ci; GRANT ALL PRIVILEGES ON spam_db.* TO 'spam_db_user'@'localhost' IDENTIFIED BY 'spam_db_password';

MySQLdb

• MySQLdb provides the Python bindings to interface with the database.

  1 2 3

  curl -O http://internap.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.2.tar.gz tar -xvzf MySQL-python-1.2.2.tar.gz cd MySQL-python-1.2.2

• Edit the site.cfg file:

  • Uncomment the line that begins #mysql_config =

  • Edit the path to point to /usr/local/mysql/bin/mysql_config:

    mysql_config = /usr/local/mysql/bin/mysql_config
    

• Complete the build and installation:

  1 2 3

  python setup.py build sudo python setup.py install cd ..

PIL

The Python Imaging Library adds image processing capabilities and supports many file formats. Probably on of the most common applications I use it for are resizing and cropping images, and generating thumbnail images. This installation seems to cause many people grief, particularly JPEG handling (which is an optional install that depends upon libjpeg).

• freetype (optional) - a software font engine

  1 2 3 4 5 6 7

  curl -O http://download.savannah.gnu.org/releases/freetype/freetype-2.3.6.tar.gz tar -xvzf freetype-2.3.6.tar.gz cd freetype-2.3.6 ./configure make sudo make install cd ..

• libjpeg

  1 2 3 4 5 6 7 8

  curl -O http://www.ijg.org/files/jpegsrc.v6b.tar.gz tar -xvzf jpegsrc.v6b.tar.gz cd jpeg-6b ./configure make make test sudo make install-lib cd ..

• PIL build

  1 2 3 4

  curl -O http://effbot.org/media/downloads/Imaging-1.1.6.tar.gz tar -xvzf Imaging-1.1.6.tar.gz cd Imaging-1.1.6 python setup.py build_ext -i

  Checking for the following lines near the end of the output from the previous command:

  --- TKINTER support ok
  --- JPEG support ok
  --- ZLIB (PNG/ZIP) support ok
  --- FREETYPE2 support ok
  

  Complet install and perform a last sanity check:

  1 2 3 4

  python selftest.py sudo python setup.py install python -c "import Image; Image.open('Images/lena.jpg').show()" cd ..

  You should see a Preview window pop up with the picure of a woman wearing a hat.

Common Packages

• IPython - An enhanced Python shell. You will want it - and be lost without it very quickly. Django will look for IPython when you run the shell command extension.

  1 2 3 4 5 6 7

  curl -O http://ipython.scipy.org/dist/ipython-0.8.4.tar.gz tar -xvzf ipython-0.8.4.tar.gz cd ipython-0.8.4 python setup.py build sudo python setup.py install python setup.py install_scripts --install-dir=/usr/local/bin cd ..

• docutils

  1 2 3 4 5 6

  curl -O http://internap.dl.sourceforge.net/sourceforge/docutils/docutils-0.5.tar.gz tar -xvzf docutils-0.5.tar.gz cd docutils-0.5 python setup.py install sudo cp tools/buildhtml.py tools/rst2html.py /usr/local/bin cd ..

• Markdown

  1 2 3 4 5

  curl -O http://internap.dl.sourceforge.net/sourceforge/python-markdown/markdown-1.7.tar.gz tar -xvzf markdown-1.7.tar.gz cd markdown-1.7 sudo python setup.py install cd ..

Django

Installation of Django, using either Apache with mod_python, mod_wsgi, or FastCGI has been written about in numerous locations. However, here is my personal strategy for managing Django across various revisions:

• First, I set up a directory to store the revisions of Django that I need across the sites I manage:

  1 2	mkdir ~/python/django pushd ~/python/django

• Next, I have two helper scripts I use to manage my revisions:

  A revision "getter" called djget:

  1 2 3 4 5 6 7

  #!/bin/bash svn export -r $1 http://code.djangoproject.com/svn/django/trunk $1 pushd $1/docs buildhtml.py --local mkdir html mv *.html html popd

  Run djget to export your desired revisions (why not just to stick to one revision? I manage a few sites and not all are on the same revision. Also, I like to play around with the trunk tip on my local machine first before deploying it live).

  Example:

  djget 7728
  

  This will export the desired revision into a directory named for the revision number (7728, in this case). It also generates HTML files from the reStructuredText documentation and moves those files into a new html directory

  A revision switcher called switch:

  1 2 3 4 5 6

  #!/bin/bash if [ -e django ]; then rm django; fi if [ -e html ]; then rm html; fi ln -s $1/django ln -s $1/docs/html

  Example:

  switch 7728
  

  This deletes existing symlinks to another revision and recreates them accordingly.

• I add the html symlinked directory to my Bookmarks Bar in Safari

• Finally, I symlink the django symlink in my local site-packages:

  1 2 3

  pushd ~/python/site-pacakges ln -s ~/python/django/django popd

Optional packages

• extensions - a repository for collecting global custom management extensions for the Django Framework. As a contributor to the repository, I would be remiss I neglected to mention it.

  1 2 3 4 5

  curl -O http://django-command-extensions.googlecode.com/files/django-command-extensions-0.3.tgz tar -xvzf django-command-extensions-0.3.tgz cd django-command-extensions-0.3 sudo python setup.py install cd ..

  - or -

  An example of how I use symlinks in my local site-packages to handle scenarios where I am frequently updating packages:

  1 2 3 4

  svn co http://django-command-extensions.googlecode.com/svn/trunk/ django-command-extensions pushd ~/python/site-packages/ ln -s ~/python/install/django-command-extensions/extensions popd

  To use in your projects, just add 'extensions' to your INSTALLED_APPS settings.py file (details).

• Pygments - a generic syntax highlighter for general use in all kinds of software such as forum systems, wikis or other applications that need to prettify source code

  1 2 3 4 5

  curl -O http://pypi.python.org/packages/source/P/Pygments/Pygments-0.10.tar.gz tar -xvzf Pygments-0.10.tar.gz cd Pygments-0.10 sudo python setup.py install cd ..

• mdx_codehilite - markdown code highlighting extension can be installed as well (requires Pygments):

  1 2 3 4 5 6

  curl -o mdx_codehilite-0.2.tar.gz \ http://gitorious.org/projects/python-markdown-extra/repos/codehilite/\ archive/f30802583958c9f54e8b4390d8dafa8dd4a147fa.tar.gz tar -xvzf mdx_codehilite-0.2.tar.gz cp -v python-markdown-extra-codehilite/mdx_codehilite.py \ /Library/Frameworks/Python.framework/Versions/Current/lib/python2.5/site-packages

  mdx_codehilite is used to added syntax highlighting markup to this blog

• dateutil - provides powerful extensions to the standard datetime module

  1 2 3 4 5

  curl -O http://labix.org/download/python-dateutil/python-dateutil-1.4.tar.bz2 tar -xvjf python-dateutil-1.4.tar.bz2 cd python-dateutil-1.4 sudo python setup.py install cd ..

  
  

• path - provides a class (path) for working with files and directories (less typing than os.path)

  1 2 3 4 5

  curl -O http://pypi.python.org/packages/source/p/path.py/path-2.2.zip unzip path-2.2.zip cd path-2.2 sudo python setup.py install cd ..

  I've used the path package for years, but many people are adopting Unipath

• twill - a scripting system for automating Web browsing. Useful for testing Web pages or grabbing data from password-protected sites automatically.

  1 2 3 4 5

  curl -O http://darcs.idyll.org/~t/projects/twill-0.9.tar.gz tar -xvzf twill-0.9.tar.gz cd twill-0.9 sudo python setup.py install cd ..

• feedparser - Parse RSS and Atom feeds in Python

  1 2 3 4 5

  curl -O http://pypi.python.org/packages/source/F/FeedParser/feedparser-4.1.tar.gz tar -xvzf feedparser-4.1.tar.gz cd feedparser-4.1 sudo python setup.py install cd ..

Miscellany

• Here are a few aliases I have set in my ~/.bashrc file for developing on Leopard:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

  #!/bin/bash # System helpers # Flush DNS - after making changes to your `/etc/hosts` file: alias flushdns='dscacheutil -flushcache' # List of open ports alias ports='sudo lsof -i -P' # Python helpers: alias site-packages='cd /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/' alias pyclean='find . -name "*.pyc" -exec rm {} \;' # Django helpers: alias runserver='python manage.py runserver' alias runserver0='python manage.py runserver 0.0.0.0:8888' alias djshell='python manage.py shell' # TextMate helpers: export SVN_EDITOR="mate -w" export EDITOR="mate -w"

Part one in my new series on some of the finer libations I've had the fortune to sample in past year or so.

Price Key

These prices are roughly what I pay in New Hampshire and tend to be about 10% pricier in Massachusetts. Shocking, I know. Your mileage may vary:

• $ $10-15
• $$ $16-20
• $$$ $21-25
• $$$$ $26-30

The Wines

1. St. Supery Cabernet Sauvignon $$$$

   To me, this Cab tastes as good as others I've had at twice the price. Better yet, you can occasionally find half bottles (375 mL)

2. Layer Cake Shiraz $$

   "Layer Cake" exploded on the scene last year (as far as I know) and sold out very fast. Of course, that means the prices went up this year.

3. Penfolds Bin 389 Cabernet Shiraz $$$

   So far, I've yet to try a Penfolds that I didn't like. All of the "Bins" are fantastic.

4. Ruffino Riserva Ducale Oro Chianti Classico $$$

   At one time I would have listed more Chiantis - or at least more Italians - here, but my tastes have run more to the fuller reds these past few years.

5. Charles Heidsieck Brut Reserve Champagne $$$$

   I think I like this one better than Vueve Cliquot, and its a third less expensive (but nearly impossible to find in New England).

6. La Crema Pinot Noir $$

   The only Pinot on my list. I tend to like earthier Pinots. I also tend to think any Pinot priced under $20 probably isn't going to be worth it and the "La Crema" hovers right around there, so it's a win-win scenario by me.

7. Clos Du Bois Cabernet Sauvignon $

   Probably the cheapest entry on the list, yet the best choice for an "any" day table wine. I wait for the sales that come along every other month or so and save an additional few bucks.

8. Jester Shiraz $$

   The best screw cap wine I've yet to try.

9. Beaulieu Vineyard (BV) Napa Valley Cabernet Sauvignon $$

   The BV's are are very nice, easily found, and get real yummy when the move up into the next category. "Tapestry" is one to look for in the pricey range.

10. Hawk Crest Cabernet Sauvignon $

    See "Clos Du Bois". Put out by Stag's Leap, this is another one that goes on sale frequently up here in New Hampshire.

11. Domain Carneros Brut Sparkling Wine $$

    The most Champagne-like of the sparkling wines, this one tastes like it should be double the cost.