NOTE: This post is for installing Python 2.4 on a brand new Snow Leopard Instance. If upgrading on top of Leopard, you may have to update easy_install, macports, etc.  More Googling around may be required.

Though I had to create two buildouts to get this to work — is there a way to get this into one buildout?

I first tried to create one buildout by combining  Florian Schulze’s buildout recipe with a standard Zope recipe — but since initial bootstrap was run by Python 2.5, I couldn’t get the Zope instance to use the new Python 2.4. So I first ran a buildout to build Python 2.4 (using OSX-installed Python 2.5), then used that new Python 2.4 to run bootstrap.py on the Zope 2.8.x buildout.

Here’s the recipe I used to just build Python 2.4 (requires Florian’s buildout, see Alexander Limi’s comment below for where to find this):

[buildout]
#extends = src/snowleopard.cfg     # no longer required as Joe mentions below
python-buildout-root = ${buildout:directory}/src
parts -=
   ${buildout:python25-parts}
   ${buildout:python26-parts}

[install-links]
prefix = /opt/local

Then I ran a simple Zope 2.8 buildout to see if it would compile (using new Python 2.4 to bootstrap), and it did!

[buildout]
parts =
   zope2
   instance

[zope2]
recipe = plone.recipe.zope2install
url = http://www.zope.org/Products/Zope/2.8.9.1/Zope-2.8.9.1-final.tgz

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 8080
debug-mode = on
verbose-security = on

Here is my initial post:

I have to say — most everything I’ve installed on a fresh Snow Leopard install has worked flawlessly and swiftly — except for (the minor inconvenience of) iStat not working.  UPDATE: iStat 2.0 is available for Snow Leopard now. There’s a new beta of MenuMeters too for Snow Leopard.

There are also nice subtle improvements, see Mac Life’s 100 Top Snow Leopard Tips, Trick and Features for improvements to Preview, Expose, Stacks, etc.  I’ve very happy with the upgrade.

Now for the bad news for those like myself who depend on Python 2.4 for Plone, since many versions of Zope require Python 2.4.  I also use Python for Django, though that should run fine on Python that shipped with Snow Leopard.

You can read many of the initial details around the web, but here’s what I’ve experienced and have been able to put together:

• Note that these details are for a fresh Snow Leopard install – there are a different set of issues if you’re upgrading over your existing Leopard.  NEW: “Clark’s Tech Blog” has a nice write-up about upgrading Python after upgrading Leopard to Snow Leopard.
• Snow Leopard ships with Python 2.5.4, and this runs as a 32-bit application.
• I also need 2.4 branches of Python too, so I tried rolling my own (as usual) and it didn’t compile.  I then followed that thread for awhile.
• I then thought I pulled a fast one when I compiled from MacPorts and everything ran great!
• … but then I compiled Zope, and attempted to run an instance.  I saw a mysterious “No such file or directory” error.  Hmmm, I can navigate to that file, but running the script with my new Python interpretor was causing this error.
• After digging around with Activity Monitor, I discovered that the Python I built from scratch was running as a 64-bit app — while the Python that comes with Snow Leopard was only running 32-bit — which is telling, since most everything else on Snow Leopard is running 64-bit.
• Guessing that the the mysterious “No such file or directory” (when the file and directory did indeed exist) was due to a weird cocktail of 32-bit pieces living with 64-bit pieces.
• My latest theory was that I needed to figure out how to build Python as 32-bit.  I played with Macports and various architecture settings to hardwire this, but long-story-short — the architecture override isn’t used everywhere — so parts still compile natively as 64-bit on Snow Leopard.
• The best thread on the topic (that’s steadily growing) is here: http://bugs.python.org/issue6802 with msg92153 left today, which basically offers some additional settings for compiling Python as a 32-bit app (for Python 2.6).  Also mentions that Snow Leopard did some magic to get Python 2.5 working as a 32-bit app.
  
• My hope is that once “32-bit” Python 2.4 happens, the rest of the Zope install, etc, will be back to the good ol’ days in Leopard.

Plan B’s:

Otherwise, to save some headache, I’m wondering about installing a small Linux distro on VMWare as a local mini web-server where I can easily install Python and Zope — though that’s a bit of a pain too.

Luckily I also have my old Leopard in a separate partition (see my Extra life for my MacBook Pro with Snow Leopard and inexpensive hardware blog entry) and can boot that if necessary to work on various Zope/Plone sites (that required Python 2.4) while this is all being sorted out.

Now time to see if I can get 32-bit Python 2.4.6 compiled and installed, while waiting for more patches and information to appear…

Optimization Tutorial

I then got on my geek at “Faster Python Programs through Optimization” (Mike Müller of Python Academy), where we dove deeper into profiling and tips on improving speed or saving memory.

Some paraphrased guidelines to consider before you start optimizing (which were also reinforced in the “Real World Django” tutorial which I’ll chat about next):

• Make sure your program is really too slow – could be other factors like network traffic, database, etc.
• Don’t optimize as you go – might ultimately not need to spend that time.  Also working code is always important first.
• Only consider realistic use cases and user experience

We played with the profiling tools (profile, cProfile, time, pystone, heapy) and used them to compare various techniques:

• xrange and also Generators shaved off time by not having to allocate memory for large data sets.
• use built-in types as much as possible (including some newer collection classes)
• iterating and appending strings by first appending to lists, then using a join statement to create large strings (versus building strings via += and loops)
• One new one for me was converting lists to Sets before testing for membership of an item in the list, which is fast due to Set optimizations.
• The tutorial also covered pysco, processing and numpy modules, as well as caching techniques.

Real-world Django Tutorial

This very aptly named presentation by Jacob Kaplan-Moss and James Bennett was excellent for those of us who develop and deploy Django websites.  The full skinny (with link to slides) is here: http://jacobian.org/speaking/2009/real-world-django/

Some highlights for me included:

• Focus on tight Django Applications that promote reuse while also breaking a website into components. Benefits of also leveraging packaging up your own components.
• Gain flexibility by leveraging Django Managers, and they help encapsulate behavior behind an API.
• Can extend models via new (in Django 1.1) Proxy subclasses.
• Lots of discussion and recommendations for testing — from unit testing, through functional testing, and then browser-based functional testing. Yep, you need them all. I’d like to play more with Twill and Windmill.
• Automating deployment – including options like virtualenv (and virtualenvwrapper), Ian Bicking’s pip (”pip installs packages”), zc.buildout, and Fabric.  zc.buildout’s power was emphasized (with its recipes, etc) was a bit overshadowed by comments on lack of documentation. I’d like to give pip and Fabric a try.
• Apache + mod_wsgi is now a preferred platform for server Django sites (or at least much more consistent performance and memory-usage wise than Apache + mod_python).
• Definitely flip through the session slides!  These were just some highlights for me out of 189 slides of useful information.

Various tidbits for the next few days here at PyCon:

• Open Space sessions come highly recommended
• There’s a heavy testing thread throughout conference (10 sessions worth!)
• Friday 11am: Using Windmill
• Saturday 4:15p: Ian Bicking’s session (creator of PIP and virtualenv, among many other topics)
• Sunday 10:35a: Panel: Functional Testing Tools in Python
• … though it will ultimately be tough to pick and choose from all the great topics!

Time for some sleep… more tutorials tomorrow, then 3 days of conference, then 4 days of sprints!

ps: It’s been great to see familiar faces from the Zope and Plone communities, which is often where I “get my Python on”.  Lately I’m also doing a lot of Django, so enjoying all the synergy around Python here at PyCon2009!

Both the Interra Project and Center for Ethical Leadership are clients (and friends) of ours at Web Collective and we’re impressed with the great work they are doing.

We developed websites for them in Plone, including Boston Community Change, Puget Sound Community Change and Center for Ethical Leadership.

This award “celebrates Puget Sound leaders whose vision, commitment and unceasing efforts are significantly advancing social, environmental, and economic justice.”

Tickets are available for the 4th Annual Bill Grace Leadership Legacy Awards Dinner on March 5th, 2009.

The Interra Project “provides consumers with the incentives they need to shift their purchasing habits to support the health of their communities by shopping with locally-focused, environmentally and sustainably-minded businesses.”  I was part of the team that built the Boston Community Change website.  Now there is a  Puget Sound Community Change in our local community.  You can sign-up for a community change card for free and use when purchasing at local businesses, or if you own a business you can offer your services to those who have cards.

My experience with Neighborhood House is through a monthly Multicultural Committee meeting where I live (at New Rainier Vista) where they provide language translation and other great community services. Our community has native speakers from Somalia, Ethiopia, China, and Vietnam.

I’m now using Remember the Milk as my primary GTD tool with the Firefox extension for Gmail. The extension lets you see all your action items on the right side of the Gmail page (grouped by day) and lets you add new ones and roll-over existing ones and make quick changes and reschedules.

Its most useful feature is that you can add action items automatically when you “star” (or tag) a Gmail message.  With many of my actionable items and reminders coming in through email, this is a killer feature for me.

For example…

…if a client sends me an email that I need to respond to later in the day, I just star it and it registers as an action item for today.  If I decide I really could answer it tomorrow, I can roll over the new action item, and click the “postpone” button or type in a due date of “tomorrow”.

Then, when tomorrow comes and it’s time to act on this, I roll over the action item and now see a little “envelope” icon which allows me to jump to the email, reply to it, unstar it, and the task disappears because it’s done.

There are some very nice OSX GTD apps (like OmniFocus, and Things) but after personally jumping from client to client, I decided I wanted to go with a web-based service that had an API in case I wanted to extend it with a new UI, or maybe integrate it with another web service.

I looked at many web-based tools, but liked Remember the Milk for its API, its web-based interface (not flashy, but useful), no syncing worries, the Gmail Firefox extension (my main interface for using it), and it even has a nice iPhone web interface (with a native iPhone app rumored).  For the iPhone interface, they ask you to join their Pro membership, but $25 for a year is well worth supporting this great service.

I also built CollectiveGTD for Plone, but have decided to take a different direction with this product.  Instead of it being a tool to manage all my general GTD needs, it will evolve into a set of tools for managing Plone content within a site.

A recurring conference theme for me centered around agility. Plone has benefited by moving from Zope2 to the more flexible Zope3 Component Architecture.  Zope technologies are being made available as middleware for use on other platforms (via Repoze and WSGI). Grok is a Zope3 framework for projects needing something more lightweight. There were several very useful sessions on Agile Project Management.

There are also continuous improvements making Plone more agile for end-users.  Plone 4.0 aims to make it even easier for end-users to manage their content (including better media support, unified widgets for easier page compositioning, “page centric”, kupu improvements around layout), as well as more through-the-web features done right!

Tuesday Arrival

On my Tuesday Amtrak ride to DC, I connected with some conference folks on Twitter, found my hotel and ended up at the Science Club for a few drinks and mini-reunions. Hotel Harrington wasn’t fancy, but clean, had wifi and was only two blocks from the conference and metro, so I was happy.

Wednesday

“10% Manifesto” was my first session. In Seattle, we’ve been holding a weekly Open Source Friday, which has a lot of parallels to discussions in this session.  We’re all practicing ways of putting that extra effort in to give back to the Plone community in which we all flourish, and also socialize and learn in the process.

I then caught up on the latest happenings with Ecommerce in Plone with the GetPaid project.  This is now being used for a variety of use cases, and its component architecture seems to make it relatively straight-forward to create modules for your own needs.

“When Software is a Service, Will Only Network Luddites Be Free” brings up an interesting question: What does a free [as in freedom and open] computing cloud look like? http://autonomo.us

Thursday

I started the day as a member of the “So you want to be a Plone consultant?” panel.  It’s a big discussion for a short amount of time, but we had a nice variety of panelists and good questions. Geir Baekholt’s discussion of Open Scope contracts peaked my curiosity.

One of the questions asked “How do project come into Web Collective?”  Clients often find us through our existing work with many progressive businesses and non-profits, our contributions of software and presentations in the open source community and Seattle Plone user group, and an interest in working with a company that practices a cooperative business model where all the members are owners of the company.

I was pleasantly surprised by a several tracks on Agile Project Management (”What makes a great development team” by Mike Robinson, and “High performance teams: What’s the secret sauce?” by Gerry Kirk on Friday).  Communication and project management often determine the success or failure of a software project, and tips and tricks in this area are always welcome.

Another interesting session was “Using Grok to walk like a duck” which covered several component architectures (from the limits of subclassing all the way through to the beauty of Adapters) and how Grok (and Zope3) support a dynamic version of Adaption.

After hearing about Deliverance 6 months earlier, “Using Deliverance to theme a website” was a nice way to catch up with the latest happenings here.  If you want to apply themes to multiple sites, this seems the way to go.

Friday

Experiencing some of the complexities and limitations of formlib, I wanted to check out Stephan Richter’s “z3c.form” session. z3c.form seems more refined, and can be used in in viewlets and portlets.

After talking with Jim Fulton at the Plone Summit at Google several months ago, I’ve been anxious to play with ExtJS, but haven’t had an opportunity.  Godefroid Chapelle showed us a rich browser solution using ExtJS and KSS.

“Repoze.bfg: A zope explosion” – Several months ago, we had the pleasure of bringing Chris McDonough to Seattle to talk about Repoze.  It was pretty new back then, and WSGI wasn’t something I had played with.  Since then, and after working with some other platforms such as Django, I was excited to catch up with the latest on this project.  The primary concept is to make Zope technologies available as middleware which could mix’n'match with other Python technologies via WSGI pipeline. In addition to the nice lightweight infrastructure, I think this highlights a longer term vision of being able to share best-of-breed Python technologies across various Python frameworks.

Saturday and Sunday Plone Sprints

Record turnout of people at the Plone sprints!

I was happy to collaborate for a couple of days with Aaron VanDerlip on Plone4Artists Calendar.  I had used this on a recent project, and wanted to help make improvements to what’s becoming a defacto solution for Plone calendaring, including an implementation of recurring events.  I’m relatively new to the project and have been impressed with the product’s architecture and its recent 2.0 alpha release.  I enjoyed lending a hand to help fix bugs and refining some functionality.

I also though the concept of a “Beginner Sprint” was ingenious and helpful for including new people in the community and getting them up and running with everything they need. Nice work Jon Stahl and Joel Burton (and other Plonistas that participated).

Monday, Heading Home

Then a perfect ending — I lucked out on a direct flight to Seattle (pays to get to the airport early) and ended up sitting right next to my friend and fellow Plone collaborator Andrew Burkhalter.

This is a Plone product that implements an Open Source version of the Getting Things Done (GTD) methodology.

New in this release:
This version fixes all known unicode issues, so that non-ascii characters can now be used throughout interface for editing, viewing, and KSS inline editing.

For download, please visit http://plone.org/products/collectivegtd-thoughts/releases/1.0b2

Change log:

• NEW: Fixed all known unicode issues which were causing context/project tagging to break when editing and when viewing, and were preventing KSS from updating the UI. Language-to-date features are still only supported in English.  Templates have not been internationalized.
  • Future Plan is to replace home-grown language-to-date functionality with the parsedate python module, which also supports multiple languages.
• NEW: Added new getAllActiveActions(outputFormat=”xml) method to spit out all ActionItems, Context tags and Project tags for use by a desktop application.
• INCOMPLETE: Started to build an iPhone interface. Partially complete. Further work on this postponed.

We’ve spent the last two years forming and running a Cooperative Business, and hope others consider doing the same.

The combination of organizing as a Cooperative, being part of Plone and other Open Source software communities, and partnering with clients who share our values, have led to a strong year of collaboration and growth.

Here is a brief list of incentives and highlights for others thinking about alternative business models that may include Cooperatives and Open Source software.

Web Collective, Inc recently celebrated its one-year anniversary as a Washington State Cooperative Business.

This was challenging in that it was my first KSS project, plus most of the KSS editing examples I’ve found deal with editing just one piece of content (instead of multiple rows).

Here is a screencast to whet your appetite for inline editing. Basically, CollectiveGTD has many rows of Action Items (each with Name, Start Date, Due Date, etc) that needed to be edited.

Screencast: Example of Inline Editing in CollectiveGTD

Here were the issues that needed to be solved:

• How do we capture KSS events for a specific field in a specific row?
• How do we display inline editing forms?
• How do we handle the same row of content if it shows up in multiple places? e.g. CollectiveGTD also has a portlet which might display the same content in both the multi-row form

Here are highlights, issues and lesson’s learned:

Note: If you generally want to see all of the new KSS code in one place:
svn diff -r59525:60742 https://svn.plone.org/svn/collective/CollectiveGTD/trunk

1. Converting from a plain old HTML form to a nice KSS template

Prior to adding KSS, I started with a single HTML form that allowed users to check “Done” or “Starred” fields and click “Submit” to update all the pieces of content. The goal was to replace that with KSS, plus add inline editing. To do this, I had to remove the one HTML form since you can’t have mini-inline forms inside of a big form.

Issue: Not sure how to degrade this functionality nicely if someone doesn’t have JavaScript enabled.

I recommend creating macros for each field you’re editing, and a unique naming scheme for your inline fields.

I created 4 TAL macros (in collectivegtd_kss.pt) – one for each field to be edited with KSS. There is a lot of HTML to put in, so doing this simplifies the main page template which then just generates the page and repeats through multiple rows of data (actions_view.pt).

The first goal was to uniquely id each editable field, even if the same piece of content shows up in more than one place on the page.

To do this, each field’s CSS ID was based on the content’s short-name (to represent the specific piece of content, aka “row”) with a ‘prefix’ string (to represent the specific fieldname in case it appears in many places). For CollectiveGTD there are 4 prefixes for the main page (main_startdate, main_duedate, main_complete, main_flag) and 4 prefixes for the portlet (portlet_startdate, portlet_duedate, portlet_complete, portlet_flag).

Issue: A period in the field’s CSS ID breaks KSS. A period can appear because the short-name for a Plone piece of content can contain periods, and I’m using the short-name as the unique identifier. I haven’t fixed this one yet.

The macros require two pieces of information: ‘actionItem’ and ‘prefix’. ActionItem is a single piece of content that has all the fields a row needs. These prefix also allowed the KSS code to be able to update multiple fields if the same piece of content appears more than once. e.g. Changing Start date on the form, should also update the Start date for that items if it also appears in the portlet.

2. Inline Editing Issues

The initial tricky part of multi-row data (specifically with inline-editing) is that you can’t dynamically create KSS events for every row (nor would you want to!), so you need to structure your HTML and CSS a specific way so that when you trigger a KSS event it is smart enough to know which fields are connected with that row. My first version ended up showing and hiding all the rows forms instead of just the one I wanted.

How to hide and show the inline form:

Since inline text editing requires grouping several elements together, it’s important to group everything in one tag (e.g. <div>) and make sure all the KSS selectors for that inline form starts with that tag. The reason is that you want the parent of all the forms to be the same, so that when your KSS action fires, you can capture it within that form — and not all forms. You do all this magic via the “parentnode” function in your KSS selector.

I also set each form to have these two classes to start with (editform hiddenform) so that all the form default to being hidden.

Here’s the example of the “Due Date” field:

3. What happens after submitting an inline-edit form?

New ActionItemListKSS view for handling the server-side content updates

I created a new ActionItemListKSS (which subclasses PloneKSSView) to provide the server-side code to update content, and also reflect the changes back to the webpage (via those unique IDs we generated in the templates).

The unique IDs are used by our ActionItemListKSS.

The methods of my view are connected to the form via KSS (in collectivegtd.kss). The major methods include: update_dueDate(), update_startDate(), update_complete(), update_flag(). This is where a lot of code duplication happens (similar code in all 4 methods), but easy to understand.

They all basically are passed a ActionItem ID (content short-name) and the new value (via KSS magic). The method then updates the piece of content, reindexes the content to make sure the catalog is updated, then updates the webpage by doing an HTML replace for both prefixes (”main” and “portlet”). It the content only exists in one place, the other replace is ignored. Finally, a Portal Message is sent to the user to let them know what was changed. The two “inline edited” fields are naturally more complex.

Issue: This still seems like too much duplicate code is needed in ActionItemListKSS. Once I had one field working, I found myself refactoring common code as I learned KSS and discovered which code to reuse across fields. The refactoring is leading toward a view class useful for inline editing though!

Lesson Learned: I ended up adding helper functions in ActionItemListKSS to handle the individual field-level updates of content, because updating fields like Start Date and Due Date were complex because of the complex date conversion code — converting strings to dates, then converting dates back to strings. This was largely because I didn’t build individual field update code in CollectiveGTD. I started with formlib code which created new ActionItems all at once. In hindsight, it would be good to have a view that allows updating of each individual field — then it could be reused in both formlib and KSS without having to duplicate code. I still may do this refactoring at some point.

4. CSS file (collectivegtd.css)

The CSS was straight-forward. Minor note: The only reason some of the selectors have a lot of options is that I was playing with CSS rollovers (where one image has both rollover states) and found some boilerplate code that looks like it works for all browsers imaginable. I’m sure some of those could be removed.

5. Tests

I added tests for the ActionItemListKSS methods, but haven’t tried UI testing yet.

Conclusion

Therefore, thus…

* It’s worth learning KSS because of the power it brings you without having to write JavaScript code directly.

* The tricky piece was learning how the various pieces work together (KSS, PloneKSSView, and figuring out best-practices for structuring HTML and CSS), though once that is understood, it was easy to maintain the pieces separately by breaking the templates up into macros (to keep things simple and reusable), put all the server-side code in one Python class, with one KSS file that connects the two.

I hope this example provides you with what you need to avoid some of that initial learning curve.

I’m also interested in learning from KSS gurus if there is a way to simplify any of these pieces.

This beta release of CollectiveGTD for Plone 3.0 includes new features such as:

• AJAX Inline Editing using KSS
• Filtering of Actions by Context tag and Project tag
• An Import script that will import an iGTD CSV export and which is easy to modify for importing other formats.

The 8-minute screencast is available here: CollectiveGTD Beta 1 Screencast

The project page and link to the code is here: http://plone.org/products/collectivegtd-thoughts

Most were familiar with just one platform or the other, so it was a great opportunity to ask questions and learn about the strengths of each platform.

http://presentations.webcollective.coop/django-plone