We’ll be discussing “What did we learn at DjangoCon?” at this Thursday’s Django Seattle. See http://www.djangoseattle.org for more details.

In the meantime, here are some high-level take-aways:

• Should JavaScript and RESTful services be part of the Django core?  JS is even more useful/powerful with the latest fast JS engines in Chrome, Firefox and Safari/Webkit. Competitor Rails builds in RESTful features – some promising ones for Django include django-piston and django-roa.  I liked how Ted Leung talked about “science experiments” and posed many ideas on what we may want to experiment with to get right before approaching Django core.
• Git – Though this doesn’t directly relate to Django, DVCS systems like Git and Mercurial are in wide use.  SVN is a given, but now feel I need to know Git and Mercurial well – since popular projects are using these.  I also wanted to pick a “pet” DVCS to use as my default too.  I’ve chosen Git (mainly because of git-svn and GitHub), but will be using Mercurial as well.
• Django Tips and Tricks – Many to pick from, but I liked Query.as_sql() method to show the SQL the Django ORM generates on your behalf, the flexibility of using “signals” to loosely couple functionality (see django-signals-ahoy on bithub), reusing other Python WSGI middleware (such as repoze.bitblt, repoze.squeeze, repoze.profile), pylint/djangolint, class-based views, db schema migrations with South, much faster test speeds in Django 1.1, various test utilities floating around, talks on performance, etc.

* Django jobs are growing, and Django also a popular platform for Start-ups.

* Check out the DjangoCon2009 Wiki for slides and presentations.

NOTE: This post is for installing Python 2.4 on a brand new Snow Leopard Instance. If upgrading on top of Leopard, you may have to update easy_install, macports, etc.  More Googling around may be required.

Though I had to create two buildouts to get this to work — is there a way to get this into one buildout?

I first tried to create one buildout by combining  Florian Schulze’s buildout recipe with a standard Zope recipe — but since initial bootstrap was run by Python 2.5, I couldn’t get the Zope instance to use the new Python 2.4. So I first ran a buildout to build Python 2.4 (using OSX-installed Python 2.5), then used that new Python 2.4 to run bootstrap.py on the Zope 2.8.x buildout.

Here’s the recipe I used to just build Python 2.4 (requires Florian’s buildout, see Alexander Limi’s comment below for where to find this):

[buildout]
#extends = src/snowleopard.cfg     # no longer required as Joe mentions below
python-buildout-root = ${buildout:directory}/src
parts -=
   ${buildout:python25-parts}
   ${buildout:python26-parts}

[install-links]
prefix = /opt/local

Then I ran a simple Zope 2.8 buildout to see if it would compile (using new Python 2.4 to bootstrap), and it did!

[buildout]
parts =
   zope2
   instance

[zope2]
recipe = plone.recipe.zope2install
url = http://www.zope.org/Products/Zope/2.8.9.1/Zope-2.8.9.1-final.tgz

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 8080
debug-mode = on
verbose-security = on

Here is my initial post:

I have to say — most everything I’ve installed on a fresh Snow Leopard install has worked flawlessly and swiftly — except for (the minor inconvenience of) iStat not working.  UPDATE: iStat 2.0 is available for Snow Leopard now. There’s a new beta of MenuMeters too for Snow Leopard.

There are also nice subtle improvements, see Mac Life’s 100 Top Snow Leopard Tips, Trick and Features for improvements to Preview, Expose, Stacks, etc.  I’ve very happy with the upgrade.

Now for the bad news for those like myself who depend on Python 2.4 for Plone, since many versions of Zope require Python 2.4.  I also use Python for Django, though that should run fine on Python that shipped with Snow Leopard.

You can read many of the initial details around the web, but here’s what I’ve experienced and have been able to put together:

• Note that these details are for a fresh Snow Leopard install – there are a different set of issues if you’re upgrading over your existing Leopard.  NEW: “Clark’s Tech Blog” has a nice write-up about upgrading Python after upgrading Leopard to Snow Leopard.
• Snow Leopard ships with Python 2.5.4, and this runs as a 32-bit application.
• I also need 2.4 branches of Python too, so I tried rolling my own (as usual) and it didn’t compile.  I then followed that thread for awhile.
• I then thought I pulled a fast one when I compiled from MacPorts and everything ran great!
• … but then I compiled Zope, and attempted to run an instance.  I saw a mysterious “No such file or directory” error.  Hmmm, I can navigate to that file, but running the script with my new Python interpretor was causing this error.
• After digging around with Activity Monitor, I discovered that the Python I built from scratch was running as a 64-bit app — while the Python that comes with Snow Leopard was only running 32-bit — which is telling, since most everything else on Snow Leopard is running 64-bit.
• Guessing that the the mysterious “No such file or directory” (when the file and directory did indeed exist) was due to a weird cocktail of 32-bit pieces living with 64-bit pieces.
• My latest theory was that I needed to figure out how to build Python as 32-bit.  I played with Macports and various architecture settings to hardwire this, but long-story-short — the architecture override isn’t used everywhere — so parts still compile natively as 64-bit on Snow Leopard.
• The best thread on the topic (that’s steadily growing) is here: http://bugs.python.org/issue6802 with msg92153 left today, which basically offers some additional settings for compiling Python as a 32-bit app (for Python 2.6).  Also mentions that Snow Leopard did some magic to get Python 2.5 working as a 32-bit app.
  
• My hope is that once “32-bit” Python 2.4 happens, the rest of the Zope install, etc, will be back to the good ol’ days in Leopard.

Plan B’s:

Otherwise, to save some headache, I’m wondering about installing a small Linux distro on VMWare as a local mini web-server where I can easily install Python and Zope — though that’s a bit of a pain too.

Luckily I also have my old Leopard in a separate partition (see my Extra life for my MacBook Pro with Snow Leopard and inexpensive hardware blog entry) and can boot that if necessary to work on various Zope/Plone sites (that required Python 2.4) while this is all being sorted out.

Now time to see if I can get 32-bit Python 2.4.6 compiled and installed, while waiting for more patches and information to appear…

Thanks to 14 Sprinters who came together on July 25, we now have a Django Seattle Website at http://www.djangoseattle.org

Some of our Django Seattle Sprinters

There was a lot of infrastructure work done at the sprint which is still in development and didn’t make it to the live site yet — but the experience of getting to know each other, and learning/sharing Django knowledge was another fine Sprint accomplishment.

Here’s a brief summary of what people worked on:

• Integrated in Blogging, Profile and Calendar functionality from django-basic-apps. Initially Pinax was explored, but had a lot of dependencies and seemed better for creating specific sites genres, but was challenging to incorporate into our existing site.
• Created a Twitter portlet that shows live #djangoseattle Tweets.
• Setup Flatpages for core content, and creating a database-driven menu
• Created a logo and initial site design and templates
• Setup Django on live server
• Used the Django Debug Toolbar while developing the site
• Some were playing with Django for the first time
• Some floated around to help diagnose problems and help those new to Django

Functionality brainstorm

Thank you Sprinters

Our sprinters (in alphabetical order) were: Andrew Beyer, Jon Callahan, Jesse Franceschini, Doug, Brian Gershon, Johann Heller, Paul Pham, Micah Ransdell, Leo Shklovskii, Trevor Smith, Jesse Snyder, Alex Tokar, Ragan Webber, Ben Wilber

Thank you Sponsors

Also a Big Thank You to our sponsors, hosts and organizers.

• Jon Callahan at Mazama Science treated all 14 of us to a tasty PCC Natural Markets lunch, coffee, drinks and snacks.
• Michael Kim at Grapevyn brought in Top Pot Doughnuts.
• Paul Pham hosted us at his coworking space Office Nomads which was a great place to have a sprint.
• Leo Shklovskii at Evoworx and Brian Gershon at Web Collective had a great time organizing the sprint.

We look forward to our next sprint!

Optimization Tutorial

I then got on my geek at “Faster Python Programs through Optimization” (Mike Müller of Python Academy), where we dove deeper into profiling and tips on improving speed or saving memory.

Some paraphrased guidelines to consider before you start optimizing (which were also reinforced in the “Real World Django” tutorial which I’ll chat about next):

• Make sure your program is really too slow – could be other factors like network traffic, database, etc.
• Don’t optimize as you go – might ultimately not need to spend that time.  Also working code is always important first.
• Only consider realistic use cases and user experience

We played with the profiling tools (profile, cProfile, time, pystone, heapy) and used them to compare various techniques:

• xrange and also Generators shaved off time by not having to allocate memory for large data sets.
• use built-in types as much as possible (including some newer collection classes)
• iterating and appending strings by first appending to lists, then using a join statement to create large strings (versus building strings via += and loops)
• One new one for me was converting lists to Sets before testing for membership of an item in the list, which is fast due to Set optimizations.
• The tutorial also covered pysco, processing and numpy modules, as well as caching techniques.

Real-world Django Tutorial

This very aptly named presentation by Jacob Kaplan-Moss and James Bennett was excellent for those of us who develop and deploy Django websites.  The full skinny (with link to slides) is here: http://jacobian.org/speaking/2009/real-world-django/

Some highlights for me included:

• Focus on tight Django Applications that promote reuse while also breaking a website into components. Benefits of also leveraging packaging up your own components.
• Gain flexibility by leveraging Django Managers, and they help encapsulate behavior behind an API.
• Can extend models via new (in Django 1.1) Proxy subclasses.
• Lots of discussion and recommendations for testing — from unit testing, through functional testing, and then browser-based functional testing. Yep, you need them all. I’d like to play more with Twill and Windmill.
• Automating deployment – including options like virtualenv (and virtualenvwrapper), Ian Bicking’s pip (”pip installs packages”), zc.buildout, and Fabric.  zc.buildout’s power was emphasized (with its recipes, etc) was a bit overshadowed by comments on lack of documentation. I’d like to give pip and Fabric a try.
• Apache + mod_wsgi is now a preferred platform for server Django sites (or at least much more consistent performance and memory-usage wise than Apache + mod_python).
• Definitely flip through the session slides!  These were just some highlights for me out of 189 slides of useful information.

Various tidbits for the next few days here at PyCon:

• Open Space sessions come highly recommended
• There’s a heavy testing thread throughout conference (10 sessions worth!)
• Friday 11am: Using Windmill
• Saturday 4:15p: Ian Bicking’s session (creator of PIP and virtualenv, among many other topics)
• Sunday 10:35a: Panel: Functional Testing Tools in Python
• … though it will ultimately be tough to pick and choose from all the great topics!

Time for some sleep… more tutorials tomorrow, then 3 days of conference, then 4 days of sprints!

ps: It’s been great to see familiar faces from the Zope and Plone communities, which is often where I “get my Python on”.  Lately I’m also doing a lot of Django, so enjoying all the synergy around Python here at PyCon2009!

As usual, Python is popular in many realms.

Who attended?

We started with quick introductions – a nice mix of folks with some traveling from Portland OR, Vancouver BC and even one from Chicago and DC.  Many folks using Python — several announcing Python job openings.  People were from various organizations such as University of Washington, NOAA, ONENW, Web Collective, NPower, LexisNexis, Microsoft, Sun, and many interesting companies I didn’t catch the names of.

Quick Highlights

We started with a lightning talk with tips on moving your code toward Python 3.0 (running Python 2.6 with -3 option; using __future__, running 2to3).

Then saw a light-weight web framework called Werkzeug – I like its idea of decorating a Python view function with its URL mapping [e.g. @expose('/') to connect a view with the root of the site].

We then heard about the ease of leveraging buildbot for testing.

NOAA started the presentations with their CAMEO Chemical modeling application, “a Pylons-based web app wrapped in a wxPython interface for desktop use.”  There were various complications making this work cross-platform on both IE and Safari, but overall successful.  Chris has high hopes for upcoming wxWebKit (which wasn’t quite mature enough at the time they were developing their app), and might consider pyQT or pyGTK for future projects.

University of Washington’s Beraber was interesting – a way to offer open source cloud computing (via a Python-based VM) by sharing your computer safely with others, and being able to run programs on many computers around the world.

After lunch, lightning talks resumed with Sphinx, an RST based system for writing documentation for your code (used for Python’s documentation).

We then saw NodeBox, “a Mac OS X application that lets you create 2D visuals (static, animated or interactive) using Python programming code and export them as a PDF or a QuickTime movie.”  I checked out their website — some cool plugins like modeling of flocks.  You could probably make some very cool desktop wallpapers with this too.

If you’d like to play with virtualization and open source, Derek Simkowiak is working on a program called “vmshell” that allows you to more easily manage virtual sandboxes.  Management of VMs was mentioned as something missing from many open source VM solutions.

Our first afternoon presentation talked about the benefits of high-level languages like Python and benefits over lower-level languages like C++ or Java.  Mark McWiggins presented good arguments for why organizations may want to consider Python over these other languages.

Sage, and its 5+ million lines of code, offers open source math modeling.  For those that need Mathematica, Magma, Maple, or Matlab power, Sage was impressive — from interacting with and showing complex math formulas in Python and Javascript, to live 2D/3D plotting, to importing the library into your own Python program and going to town.  One of its innovative features (from a web dev perspective) is writing a math function (in Python) which you want to interact with it on the web — instead of creating your own web form, you can decorate your function with @interact, which introspects the function parameters and automatically creates a web form for that function.

After having played a bit with Google App Engine, it was nice to hear a real-world experience about using this in a production project.  Web 2.0 apps can be a sweet spot for GAE, though there are differences with other traditional web development methods that may help determine if your app fits GAE or not.  I won an online O’Reilly book on this topic.

I had seen mention of Cython, but hadn’t investigated.  Cython is a way to compile your Python code in C code for major speed improvements.  It has some cool profiling features like an interactive web-based code display that uses light-to-dark color-coding to show which Python code lines are the slowest, and allows you to click on the line to see the actual C code that was generated.

The last presentation was by Sun, who are investing in Python (and other languages in addition to Java) due to their popularity by programmers.  They are also investing in Jython (adding more resources than before) to bring this up to latest versions of Python 2.x, and some work on the JVM to support languages other than Java.

Pycon

Registration just opened for PyCon 2009 (in March) in Chicago.

I plan on attending this year, hope to see you there!

Thanks!

Thank you Seattle Python and the University of Washington for hosting!

After Web Collective just finished a Django website that needed to scale to 12,000 simultaneous peak users, Flickr architect Cal Henderson’s keynote was timely and very entertaining – particularly his commentary on “serious” frameworks and essentially a comic roasting of all of the major ones.

Since most of us are building “houses and not skyscrapers” there is some question as to whether Django should introduce all of Cal’s suggestions (who is a “scale” geek working with on a very large website), but how can you argue with his three major focus areas of scale, performance and “being awesome“.

Discussions of “scale” and “Django” remind me to play with Google App Engine soon. I also just discovered Django Debug Toolbar and other interesting things reading through Cal’s blog.

Learning from other Python Frameworks (Django and Zope)

With Zope (and Plone) being the platform I’ve used for some time now, I found Mark Ramm’s perspective on how to learn from all the smart folks in various Python (and other) frameworks a good 10,000 ft reminder.

He highlights that Zope3, Turbogears, and Pylons have learned some of the lessons of old Zope2 in that they are frameworks that strive to use community components that work with each other in multiple contexts.  They also are trying not to isolate themselves from the Python web community by building functionality specific to just one platform.

He wasn’t saying Django was far afloat in these areas, but rather that Django could make some of its innovations available more easily to other frameworks (e.g. Django’s ORM, and Django’s template engine), and that it should maybe use some of the Python components from other frameworks (e.g. Django middleware and WSGI middleware compatibility).

I look forward to poking around the rest of the DjangoCon 2008 videos, such as Guido van Rossum’s keynote and others on YouTube’s DjangoCon 2008 Session’s playlist.

The dev team inventoried its skills and discovered the expected diversity of platforms, skills and languages for the 30+ back-end developers – PHP, Python, Ruby, and .NET. What strategy do we use to merge our skills and get the product developed?

I suggest Django for its easy of adoption possibilities among a very diverse technical crowd, and I provide a bit of rationale for others to consider:

• We’ve setup a Linux server which likely means LAMP development (PHP, Python, Apache, MySQL) or Ruby on Rails for the final platform this will run on.
• All developers could get setup and be running quickly on all major operating systems with a standard install of Python, Django and any SQL server one wanted, such as the quick-to-install SQLite. I plan on developing completely on my laptop, and getting code to the server via subversion updates.
• I’m a strong Python developer who spends my time primarily in Zope and Plone – and haven’t yet used Django – but there are experts in the room, and anyone who can write an object class and learn some basic Python can get something up and running – or at minimum could read someone else’s code and become productive. I gained some quick familiarity by reading a Django book on the bus ride home tonight!
• There are no technology majorities in the group — but I think everyone ramping up on Ruby on Rails, .NET (if someone brings a Windows Server) or Zope/Plone seem slim for a 2-day event. Django seems like only an hour or two away from general productivity. Plus documentation is on the web.
• PHP could also be possibility, but not sure what framework we’d use.
• Ruby on Rails may also fall into the camp of non-Ruby developers being able to hack existing code setup by Ruby on Rails experts, but I still think it may require more learning than we have.

I’ve been enjoying all the conversations, idea brainstorming, and seeing some familiar faces (such as Josh Livni and Brian Dorsey) while also meeting many new people!

What do you think? Have a look at Django here: http://www.djangoproject.com/

I look forward to seeing what direction we take tomorrow when coding commences!

And special thanks to those who stayed late to setup the server, trac and subversion.

I used this time to start development on a python library for the Getting Things Done (GTD) methodology. I’ve been practicing GTD over the last couple of years using a variety of tools, but just haven’t found the right one – my current favorite is iGTD (OSX). There are several others. I also use tools that have some parallels, like Basecamp.

The GTD apps I’ve used are missing one major feature. They’re sophisticated but not open source, which means I can’t tweak it to do what I really want, and have been waiting awhile for some fixes that I need.

The idea is to build a pure python library, which could be used by itself if desired. On top of this we plan to build a Plone 3.0 version.

Currently the Python library (in alpha, licensed under GPL3) handles core GTD tasks and some early task management. Derek Hoshiko (also an owner at Web Collective, Inc) and I have been brainstorming some ideas for a web-based multi-user version of the GTD framework.

Ultimately, GTD is great for individuals but what might a web-based version of GTD look like in a multi-user team environment, where you could also see and interact with others’ tasks? Can we use this for our internal team project scheduling, availability, task management, and status reporting? We plan to find out.

If you’re interested in collaborating on this project, please contact us.