optilude

Given that it’s the year 2007, you’d think companies would’ve cottoned on to a few cliched, cheesy, unbelievable and outright silly things that they shouldn’t try to market themselves with. I guess not. I don’t think I’m the only one who feels turned off by a company which uses one or more of these techniques:

• Taglines like Your main supplier of … or Your experts in …. Seriously, I’ve not heard about you before. You’re not my anything.
• Anything involving an explamation mark.
• Anything involving more than one explamation mark.
• Anything involving an exclamation mark and a question mark next to each other.
• Other grammatical atrocities, such as using numbers as phonetic replacements for words, or trying to spell things phonetically. JooToobe anyone?
  
• Pictures of attractive women with headsets waiting to take your call. Call centre operators are, by and large, not very attractive, and in any event I wouldn’t be able to see them. This argument counts doubly for the kind of companies that advertise exclusively in London telephone booths.
• Other images which were obviously purchased as stock photos.
• Pseudo-diversity. I used to work for someone who had a stock photo of a black hand shaking a white one. They also had one black employee. It wasn’t his hand.
• Anyone who insists on getting an email address (or worse, a contact number) before possibly divulging the price of something.
• Gyms.
• Over-use of superlatives or claims to being a silver bullet for whatever problem. I would come up with an example, but the most superb example I know of comes from Tom Wait’s Step Right Up.
• Websites which look like this. Bonus for the dancing robot though.
• Websites which don’t work in Firefox and/or Safari. “But most of the world uses Internet Explorer” is just a pathetic excuse these days.
• Companies or products which attempt to speak to customers in the first person. “Hi, I’m your friendly supplier of …” or “let me give you a hand with that”. The UK tax return is like that, and it’s damned irritating. Of course, I wouldn’t trust the taxman anyway.

Oh well. Enough procrastination for a while, back to work.

Your website is pointless. It has a stupid name. It looks lame. It contains no useful information whatsoever. Your trackback spam is annoying. Please, just cease to exist. Or at least cease to latch yourself onto every last blog entry I post.

5…4…3…2…1…

Okay, so I’m a bit biased (I wrote a bunch of it), but I had the chance to actually use Plone 3 in anger today. And it rocks. It rocks hard. It’s just more fun to use, it feels slick, it feels fancy. Y’all don’t know what you’re missing.

Go try the beta and see what I mean.

Yesterday and today, I did something I’ve been wanting to do for a long time - refactor the more generic bits of b-org out into more generic components. I have the luxury of targeting Plone 3.0 only, so I could use a lot of new APIs and components only available there. The results are:

• borg.localrole - a standalone release of the PAS plug-in which allows you to define local roles in an adapter. This will probably work on Plone 2.5, but on its own it does nothing.
• borg.project - an easy-to-use implementation of the “project workspace” metaphor.

The latter is possibly more interesting. When you install it, a manager can add a new Project, and configure a list of managers (who get the Manager role); a list of members (who get the TeamMember role); and a list of content types which project members will be able to add inside the workspace. There is also a local workflow policy (using CMFPlacefulWorkflow - watch out, you need to install this manually now in Plone 3).

This is no different to what the same part of b-org did, except:

• It’s completely standalone
• It works on any kind of user, not just Membrane ones
• It has few dependencies - it doesn’t need Membrane; in fact, it doesn’t even use Archetypes.
• It shows off a number of cool Plone 3-like features, such as add forms, formlib-based edit forms, factories, browser views, events and doctests

I know the Teamspace guys have been working on similar things (such as team spaces working with plain members). From what I know, Teamspace is more powerful and ambitious, possibly also a bit more complex. I must admit that the ability to work against plain Plone 3, starting from a clean slate was a strong driving force. I got to exercise a lot of things that we want people to be able to do - I even fixed a few Plone 3 bugs in the process, which I probably wouldn’t have found before the release otherwise.

Both projects are in the Cheesehop. Both have README.txt files with doctests and background info. If you want to see how you may code against Plone 3, or you need a simple, lightweight project workspace, give it a go!

If you just want to have a browse of the code, look in the Collective.

It’s not often that I make mistakes, but I made some in the design of Lead, specifially in its use of the Zope 3 Component Architecture (CA). I think there are some useful lessons in these mistakes, and in the way I ended up doing things. Feel free to disagree.

Mistake #1 - Inventing an arbitrary new registry

The first mistake was one of not using the CA enough. Lead allows you to set up several databases, essentially with different connection parameters. An SQLAlchemy Engine is instantiated, lazily, based on this information and then made available via a component providing IDatabase, whose job it is to give access to an SQLAlchemy Session and Connection.

My first design had the following interaction pattern:

>>> from collective.lead.interfaces import IDatabases
>>> from zope.component import getUtility
>>> databases = getUtility(IDatabases)
>>> my_db = databases['my_db']

my_db would now be an instance of IDatabase, constructed lazily the first time it was retrieved. The global IDatabases utility maintained a dict of already-constructed IDatabase’s.

Here’s how it works now instead:

>>> from collective.lead.interfaces import IDatabase
>>> from zope.component import getUtility
>>> my_db = getUtility(IDatabase, name='my_database')

This is a much more natural API - the client code is looking for a resource (a database connection) and looks it up by type (IDatabase) and name. It did mean putting the lazy Engine instantiation logic inside IDatabase rather than some factory code, but that’s code that I only had to write once.

Mistake #2 - Over-componentising

The second mistake was to over-componentise the design. Lead is concerned with the instantiation of Engine’s and the management of transactions. Applications are supposed to register a new database (by name), providing the code to construct a data model with SQLAlchemy Table’s, an ORM model with SQLAlchemy Model’s, as well as provoding the DSN for the database.

In the old design, the application was responsible for registering three (!) different utilities:

1. A named utility providing ITables. This was a dict-like mapping of tables, with a method called setUp() which was called by the IDatabases utility to set it all up.
2. Similarly, a named utility providing IMappers contained mappers, set up from the tables when the IDatabases utility called setUp() on it.
3. A named utility providing IDatabaseConnectionSettings provided the URL to use in the DSN when constructing the engine.

These all had to have the same name. The first time some client code requested a database by name from the IDatabases utility, it would look up each of these and construct an Engine, initialize the ITables and IMappers utilities and return the IDatabase.

Mostly, this design evolved because I was falling for the great CA design myth:

Component Architecture design means “don’t do subclasses”

Rubbish!

Inheritance in OOP is a fine way of modelling an “is-a” relationship. What proponents of component design suggest, is that using mix-in classes to support common features across a hierarchy of types leads to hard-to-maintain and difficult-to-extend code.

A database connection, as represented by an IDatabase utility, “is a” database. Using the general utility syntax, we can obtain one by name. All we need is for the application code to register a utility with the specific characteristics of a named database. And since most IDatabase utilities will share the same fundamental logic, it’s appropriate to provide a base class for IDatabase utilities.

Here’s the way you use it now:

from collective.lead import Database
import sqlalchemy as sa

class MyTable(object):
    pass

class MyDatabase(Database):

    url = sa.engine.url.URL(host='localhost', user='root', database='db', driver='sqlite')

    def _setup_tables(self, metadata, tables):
        tables['mytable'] = sa.Table(’sometable’, metadata)

    def _setup_mappers(self, tables, metadata):
        metadata['mytable'] = sa.mapper(MyTable, tables['mytable'])

And then you register this as a factory for a named utility providing IDatabase.

You might recognise this as the Template Method design pattern. Of course, being components, there’s nothing to say you can’t register another named utility providing IDatabase, without using this base class, so long as it conforms with its interface. The base class is an implementation detail which helps the utility writer getting the code right, nothing more.

I also used an adapter internally to represent the ITransactionAware aspect of a databsae connection, mostly to keep this out of the public API of the IDatabase class - this is an example of where using components rather than mix-in classes is probably a good idea.

So, like I said in my previous post, SQLAlchemy rocks. However, it requires a bit of hoop-jumping to sync its transactions with Zope (2) transactions, and manage the creation of engines and connections. There are at least two SQLAlchemy/Zope 2 libraries which I know about:

z3c.sqlalchemy

Provides transaction integration, as well as some abstractions for creating SQLAlchemy mappers. Mapper classes can be generated, and rely on particular base classes to provide convenience functionality

ore.alchemist

Provides transactions integration as part of a larger suite of tools centered around briding SQLAlchemy mappers and Zope 3 interfaces, supporting among other things zope.formlib forms

Of these, ore.alchemist is quite intriguing, but way more than I need right now. I was using z3c.alchemist, but it seems Andreas wants it to go in a slightly different direction than I am going: The two nails in the coffin for me were the requirement to use special base classes (which of course will provide additional convenience APIs, but I have a need to let SQLAlchemy map exisiting classes which are used elsewhere in my application, and for which I do not want explicit additional dependencies) and the lack of support for intermixing SQLAlchemy ORM sessions with lower-level connections for executing raw SQL or Python-constructed SQL. This means that you can’t use a session to save some values and then expect those rows to be returned if you subsequently perform a “raw” query.

Thus is born Lead, aka collective.lead (as opposed to gold, geddit?). You can find it in the Cheese Shop or Collective subversion repository.

Lead aims to support two things, and two things only:

• Linking SQL transactions to Zope transactions, so that there is one transaction and one connection (only opened if necessary, of course) per request and SQL commit/rollback is synced with Zope transaction commit/abort
• Making databases available as named utilities (so you can configure several databases simultaneously, e.g. from different products), instantiated in such a way that it’s possible to read connection parameters from the ZODB or otherwise determine them at runtime (in my case, with a Plone control panel form to change them)

Here is how it may look:

 class TableOne(object):
     pass

 class TableTwo(object):
     pass

Those are two “domain” classes representing database entities. We will map these to the database in a moment. Of course, you would probably have a few properties explicitly defined, and/or an interface specified.

 from collective.lead import Database
 import sqlalchemy as sa 

 class MyDatabase(Database):
     @property
     def _url(self):
         return sa.engine.url.URL(drivername='mysql', username='user',
                    host='localhost',  database='testdb')

     def _setup_tables(self, metadata, tables):
         tables['table1'] = sa.Table(’table1′, metadata, autoload=True)
         tables['table2'] = sa.Table(’table2′, metadata, autoload=True)

     def _setup_mappers(self, tables, mappers):
         mappers['table1'] = sa.mapper(TableOne, tables['table1'])
         mappers['table2'] = sa.mapper(TableTwo, tables['table2'],
                                         properties = {
                                             ‘table1′ : sa.relation(TableOne),
                                             })

This is how you set up a new database connection. The _url property needs to return an SQLAlchemy URL, which specifies which database to connect to. I fetch this from a local utility which saves connection parameters in the ZODB.

The SQLAlchemy engine will be set up on-demand (basically, the first time someone fetches a session or a connection), using the ‘threadlocal’ strategy. When this happens, _setup_tables() and _setup_mappers() are called. This is where you encode your database schema and ORM mappings. They only get called once per database utility, but then again, your database schema is unlikely to change at runtime.

You would then register this class as a factory for a utility:

    <utility
        provides="collective.lead.interfaces.IDatabase"
        name="my.database"
        factory=".db.MyDatabase"
        />

The IDatabase interface (which is implemented by the base class collective.lead.Database) has two important attributes - session and connection. These are just an SQLAlchemy Session and a Connection, but they are tied to the threadlocal connection and transaction managed by Lead.

Here’s how you might use them:

 from zope.component import getUtility
 from collective.lead import IDatabase

 db = getUtility(IDatabase, name='my.database')

 db.session.query(TableOne).list()
 ...

 db.connection.execute("SELECT * FROM table1")
 ...

Hopefully that didn’t look so hard. I’ve released 1.0b1 to the cheesehop, which works for me… you should be able to depend on it - at least, I will be.