optilude

This blog is closed - my new home is at http://martinaspeli.net.

Thunderbird upgraded itself the other day. Ever since, I’ve been rather pleased with it.

I went from Opera (too clunky and non-OSX-like) to Mail.app (too slow and no newsgroup support) to Thunderbird ages ago, and whilst it was pretty decent, it was always a little bit out of place. Now, with 2.0.0.6, it’s more harmoneously styled, snappier and has a few nice features like filtering “recent” and “favourite” folders and really useful “Back” and “Forward” buttons (otherwise known as the “oops, didn’t mean to mark that message as read and move on to the next one, where was it again?” button).

Thunderbird also has decent keyboard shortcuts and works very well as a general mail and news reader. Well done Mozilla.

In an age of YouTube, it seems that every website needs to have a little embedded video player, showing screencasts and interviews and recorded presentations. How neat. However: I just don’t have twenty minutes to watch a video just to get some piece of information I’m looking for. Ever.

Videos can’t be searched for key words. Skipping back and forth is an imprecise means of finding the information you want. I never do just one thing at a time, and videos by their nature demand full sensory attention: watch, listen, sit still.

I can see how screencasts can be useful for people who are new to a technology, say, and don’t have the means to get direct hands-on experience. A good presentation can be good on video too. But please, if there’s actual information to be presented, make it text and images first, and video  as a supplementary extra for those who can be bothered.

It’s not often that I mourn the closure of a chain of shops, but the last decent chain of record stores in the UK just went the way of the dodo. http://fopp.co.uk carries this note:

It is with great regret that we announce the closure of Fopp.

Our store chain is profitable, well regarded and loved by our loyal customers and staff. However we have failed to gain the necessary support from major stakeholders, suppliers and their credit insurers to generate sufficient working capital to run our expanding business.

We would like to thank staff and customers for their support over the past 25 years.

Any outstanding website orders have now been cancelled and will not be fulfilled or charged.

Sob… The Fopp stores had a decent amount of CDs covering a variety of styles, wasn’t outrageously overpriced, had friendly and knowledgeable staff and sold good films, books and other media. I was never able to reist going into a branch when I passed by one, and normally left with five or six CDs. I hope someone comes in to fill the void, but given how many ridiculously over-sized, over-priced Virgin Megastores and HMVs that are popping up, I won’t get my hopes up.

Just a quick note - a few people were asking about pre-ordering my book. Packt just gave me the URL for this:

Pre-order Professional Plone Development here.

And on a different note, I’m on holiday for real as of now and probably offline for two weeks.

It feels weird writing this post, having spent so long keeping it a secret (not terribly well, mind), but the time is right now, I feel, with Plone 3 RC1 just around the corner.

For the past six months or so, I’ve been writing a book which will (most likely) be entitled Professional Plone Development, published by Packt Publishing, the people behind Building Websites with Plone. It will take another month or two (or thereabouts) for it to hit the shelves, but I am scheduled to deliver the final drafts to the publisher at the end of July.

About the book

As the title suggests, this is a book for developers who want to build robust content-centric web applications on top of Plone. The ideal reader knows Python and web programming concepts in general, and has used Plone before, perhaps as a power user or site administrator. Some prior experience with Plone customisation and programming may be helpful, but shouldn’t be necessary. If you’re brand new to Plone, you should be able to pick up what you need, but spending some time playing with a standard Plone site may help you get familiar with the terminology and concepts.

This is not a book for site administrators or end users who want to use “vanilla” Plone.

It is a book about best practice. I wanted to convey the way I feel development should be done with Plone 3, without being burdened with too many legacy techniques. To my knowledge, this is the first book to actively encourage Zope 3-style patterns in Plone development and downgrade legacy concepts such as ZSQL methods and developing new persistent CMF tools.

Among other things, the book covers:

• Setting up a development environment with zc.buildout.
• Creating packages as Python eggs.
• Encapsulting site policy into a “policy product”, using GenericSetup and Python code to enable repeatable and testable deployments.
• Customising security and workflow.
• Safely incorporating third-party products into a site policy.
• Creating a new theme for Plone, customising the appearance of viewlets, portlets, Zope 3 views and page templates.
• A reference chapter outlining “Zope development concepts in a nutshell”, with runnable doctests to explain key concepts. Here’s where you’ll learn about adapters and utilities and acquisition.
• Building content types with Archetypes, demonstrating modern techniques such as using GenericSetup for configuration and keeping presentation logic in separate view classes.
• Creating standalone pages and forms with zope.formlib and Zope 3 views.
• Creating Zope 3 viewlets and Plone 3 portlets.
• Connecting to relational databases using SQLAlchemy.
• Dealing with members and member properties through PAS.
• Using KSS, Plone 3’s new AJAX framework to build dynamic pages.
• Setting up a production server with ZEO.
• Configuring caching for robust site performance.
• Connecting to an LDAP repository for authentication and member data.

There is an emphasis on proper testing, documentation, and other good practices. As far as possible, I try to explain underlying concepts and rationale - the “why” in addition to the “how”. I don’t believe in frivolous or abstract examples that skip one concept to demonstrate another. I want readers to have something they can build upon, examine and learn from, not something they have to piece together themselves. Throughout the book, a semi-realistic case study is used to build a robust application demonstrating each concept being introduced.

On writing the book

This is my first book, and it’s been a very interesting experience. First of all: it’s a lot of work. Philipp von Weitershausen, author of Web Component Development with Zope 3, warned me. I didn’t listen. In particular, developing the examples took a long time. Still, it’s pretty rewarding work, and has given me the chance to write a lot of “fun” code.

Writing a book against a pre-release version of Plone provided some particular challenges. More than once (and at least once more to come) I’ve had to make major changes to my code as something in Plone (or more often it seems, CMF) changed. I’m very grateful to the various people I’ve nagged about completing parts of Plone 3 (in particular David Convent and Wichert Akkerman) that I needed to write a particular chapter.

I’m quite convinced that Plone 3 is a lot better off because I wrote this book. For a while, I was probably the only person developing for it in anger. I’ve fixed a lot of bugs and introduced numerous small features and improvements over the past few months, as I found things that were too difficult or plain impossible to do. I found (and fixed or harassed others into fixing) a few issues with Zope and CMF as well.

Having to explain (or justify) the way something works to a reader really made me think about how good our story was in each particular area. Sometimes, it was less work to improve Plone and have a compelling story to tell, than to write about the way it worked previously.

Thank you!

I owe several people a debt of gratitude for being able to do this at all. Wichert Akerman (wiggy), the Plone 3 release manager, has been an amazing help: as a technical reviewer, in being forthcoming with the Plone release schedule, in discussing issues I felt important to address, and most recently, in teaching me about LDAP and deployment strategies, and developing tools for Plone that made these chapters much easier to write.

Jon Stahl and Andrew Burkhalter have been great reviewers as well, mercilessly coming down on my rambling style and overlong sentences. They have also been great encouragement and fed back points that they felt their clients and users may want to know more about.

Marco DeVitis has been a sharp critic (in the constructive sense) and the perfect “guinea pig” for my chapters. Chapter 3 had about 10 revisions thanks to Marco not giving up on me.

Philipp von Weitershausen provided some invaluable advice at the beginning of the project, as did Paul Everitt and Alexander Limi.

And finally, thank you to all the Plone developers who helped make Plone 3 such an incredible release. I want to work with nothing else these days.

Your website is pointless. It has a stupid name. It looks lame. It contains no useful information whatsoever. Your trackback spam is annoying. Please, just cease to exist. Or at least cease to latch yourself onto every last blog entry I post.

5…4…3…2…1…

Okay, so I’m a bit biased (I wrote a bunch of it), but I had the chance to actually use Plone 3 in anger today. And it rocks. It rocks hard. It’s just more fun to use, it feels slick, it feels fancy. Y’all don’t know what you’re missing.

Go try the beta and see what I mean.

Yesterday and today, I did something I’ve been wanting to do for a long time - refactor the more generic bits of b-org out into more generic components. I have the luxury of targeting Plone 3.0 only, so I could use a lot of new APIs and components only available there. The results are:

• borg.localrole - a standalone release of the PAS plug-in which allows you to define local roles in an adapter. This will probably work on Plone 2.5, but on its own it does nothing.
• borg.project - an easy-to-use implementation of the “project workspace” metaphor.

The latter is possibly more interesting. When you install it, a manager can add a new Project, and configure a list of managers (who get the Manager role); a list of members (who get the TeamMember role); and a list of content types which project members will be able to add inside the workspace. There is also a local workflow policy (using CMFPlacefulWorkflow - watch out, you need to install this manually now in Plone 3).

This is no different to what the same part of b-org did, except:

• It’s completely standalone
• It works on any kind of user, not just Membrane ones
• It has few dependencies - it doesn’t need Membrane; in fact, it doesn’t even use Archetypes.
• It shows off a number of cool Plone 3-like features, such as add forms, formlib-based edit forms, factories, browser views, events and doctests

I know the Teamspace guys have been working on similar things (such as team spaces working with plain members). From what I know, Teamspace is more powerful and ambitious, possibly also a bit more complex. I must admit that the ability to work against plain Plone 3, starting from a clean slate was a strong driving force. I got to exercise a lot of things that we want people to be able to do - I even fixed a few Plone 3 bugs in the process, which I probably wouldn’t have found before the release otherwise.

Both projects are in the Cheesehop. Both have README.txt files with doctests and background info. If you want to see how you may code against Plone 3, or you need a simple, lightweight project workspace, give it a go!

If you just want to have a browse of the code, look in the Collective.

It’s not often that I make mistakes, but I made some in the design of Lead, specifially in its use of the Zope 3 Component Architecture (CA). I think there are some useful lessons in these mistakes, and in the way I ended up doing things. Feel free to disagree.

Mistake #1 - Inventing an arbitrary new registry

The first mistake was one of not using the CA enough. Lead allows you to set up several databases, essentially with different connection parameters. An SQLAlchemy Engine is instantiated, lazily, based on this information and then made available via a component providing IDatabase, whose job it is to give access to an SQLAlchemy Session and Connection.

My first design had the following interaction pattern:

>>> from collective.lead.interfaces import IDatabases
>>> from zope.component import getUtility
>>> databases = getUtility(IDatabases)
>>> my_db = databases['my_db']

my_db would now be an instance of IDatabase, constructed lazily the first time it was retrieved. The global IDatabases utility maintained a dict of already-constructed IDatabase’s.

Here’s how it works now instead:

>>> from collective.lead.interfaces import IDatabase
>>> from zope.component import getUtility
>>> my_db = getUtility(IDatabase, name='my_database')

This is a much more natural API - the client code is looking for a resource (a database connection) and looks it up by type (IDatabase) and name. It did mean putting the lazy Engine instantiation logic inside IDatabase rather than some factory code, but that’s code that I only had to write once.

Mistake #2 - Over-componentising

The second mistake was to over-componentise the design. Lead is concerned with the instantiation of Engine’s and the management of transactions. Applications are supposed to register a new database (by name), providing the code to construct a data model with SQLAlchemy Table’s, an ORM model with SQLAlchemy Model’s, as well as provoding the DSN for the database.

In the old design, the application was responsible for registering three (!) different utilities:

1. A named utility providing ITables. This was a dict-like mapping of tables, with a method called setUp() which was called by the IDatabases utility to set it all up.
2. Similarly, a named utility providing IMappers contained mappers, set up from the tables when the IDatabases utility called setUp() on it.
3. A named utility providing IDatabaseConnectionSettings provided the URL to use in the DSN when constructing the engine.

These all had to have the same name. The first time some client code requested a database by name from the IDatabases utility, it would look up each of these and construct an Engine, initialize the ITables and IMappers utilities and return the IDatabase.

Mostly, this design evolved because I was falling for the great CA design myth:

Component Architecture design means “don’t do subclasses”

Rubbish!

Inheritance in OOP is a fine way of modelling an “is-a” relationship. What proponents of component design suggest, is that using mix-in classes to support common features across a hierarchy of types leads to hard-to-maintain and difficult-to-extend code.

A database connection, as represented by an IDatabase utility, “is a” database. Using the general utility syntax, we can obtain one by name. All we need is for the application code to register a utility with the specific characteristics of a named database. And since most IDatabase utilities will share the same fundamental logic, it’s appropriate to provide a base class for IDatabase utilities.

Here’s the way you use it now:

from collective.lead import Database
import sqlalchemy as sa

class MyTable(object):
    pass

class MyDatabase(Database):

    url = sa.engine.url.URL(host='localhost', user='root', database='db', driver='sqlite')

    def _setup_tables(self, metadata, tables):
        tables['mytable'] = sa.Table(’sometable’, metadata)

    def _setup_mappers(self, tables, metadata):
        metadata['mytable'] = sa.mapper(MyTable, tables['mytable'])

And then you register this as a factory for a named utility providing IDatabase.

You might recognise this as the Template Method design pattern. Of course, being components, there’s nothing to say you can’t register another named utility providing IDatabase, without using this base class, so long as it conforms with its interface. The base class is an implementation detail which helps the utility writer getting the code right, nothing more.

I also used an adapter internally to represent the ITransactionAware aspect of a databsae connection, mostly to keep this out of the public API of the IDatabase class - this is an example of where using components rather than mix-in classes is probably a good idea.