So I've settled on a name for my new template engine: Breve. Formerly known as espresso with half-and-half, Breve is now a Stan-like template engine written in pure Python with no external dependencies.

I've also written a Buffet plugin for Breve and it's been tested with Pylons and TurboGears. TurboGears testing was pretty limited (tested that a basic template rendered), but I sat down tonight and reimplemented the Pylons QuickWiki tutorial completely in Breve without issue.

I have to say that Pylons seems pretty straightforward. It's a little more upfront work than TurboGears (mostly due to Routes), but after that it's quite similar in its learning curve. One thing I like better right away is the error messages. Sometimes TurboGears exceptions can be downright obscure (and worse, occasionally being raised far away from the original exception), whereas Pylons gives almost too much information. I know which case I'd rather deal with.

Anyway, I did the Pylons test before the TurboGears test for one reason: the TurboGears site was down all day on the day I wanted to test the Buffet adapter. That meant I couldn't install TurboGears on the laptop I'm currently using. That meant no testing. That meant at least mild aggravation that TurboGears is still failing to recognize its own importance to its community. So far TurboGears has changed hosting services at least twice and still doesn't have any infrastructure to ensure availability (mirrors, anyone?). I've mentioned this to Kevin Dangoor before and I get the feeling he simply doesn't have the time to deal with it. Regardless, I'm starting to feel that simple availability might become the achilles heel of TurboGears. It depends on a couple dozen sub-projects, at least a few of which I've seen become unavailable for download at various times. The sheer number of subprojects that comprise TurboGears makes the likelihood of one of them being unavailable for install far too high. If TurboGears is going to depend on third-party packages, then it needs to do something to ensure not only its own availability, but the availability of all the sub-packages.

Anyway, I'm currently in the middle of a rather largish TurboGears application, so I won't be switching just yet, but I see Pylons in my future.

Fixed a major issue with how Breve dealt with template paths. I'm not 100% happy with the solution, but it's an issue that the frameworks (Pylons and TurboGears, in this case) tell the template engine nothing about the root path to the templates.

This is an issue for Breve because all Breve paths are relative to a static root path. I've found that this makes it far easier to have fragments in subdirectories that can still inherit from templates in higher-level directories that might then include templates from subdirectories. I tried using things like "inherits ( '../index.stan' )" with TurboStan and it turned into an unmanageable mess real quick. However, TurboStan was only really meant to support TurboGears so I cheated and called out the the TG config file to get a root directory. Supporting both Pylons and TurboGears makes that solution unusable.

Anyway, like it or not, it's been tested with Pylons 0.94 and seems to work. Note that you must specify:

breve_opts = { 'root': 'myproject.templates' }

in your config/middleware.py file.

I watched the WSGI video (and followed the slides in a second window cause the video is too tiny and compressed) and am sold. WSGI is the future of the Python web framework. While most of the major frameworks (TurboGears, Django, Twisted, et al) support WSGI, only a couple are WSGI from top to bottom: Pylons and Clever Harold. Of these two, Pylons appears to be the most mature and active, so I've decided to give it a shot. Also, the fact that the Pylons crew has ported most of the interesting stuff from Rails doesn't hurt any either.

I have to admit, I'm starting to feel like a framework whore though. Hopefully it starts to pay as well ;-)

Okay, maybe it doesn't. But I do. Fed up with Bitakora, I cracked down and started a blog from scratch (well, building on Splee's SimpleBlog tutorial - mostly for the SQLAlchemy tips).

What you see here is the first basic incarnation of it. It's built using Pylons, Breve and SQLAlchemy. Not that it's required, but it also happens to have PostgreSQL living underneath of it.

This is also my second small Pylons application (the first being the Breve site). This also makes it the second live Breve site in existence (that I'm aware of). This is actually much of the reason I'm doing it: I need to learn Pylons and I need to test Breve on real applications (I'm already seeing features that would make life a bit simpler).

Anyway, I've got a lot to do but my basic goals are as follows:

1. Very basic blog core - no Javascript. No web 2.0. Nothing but posts, comments and tags (and a few themes).
2. Plugins (for adding all the stuff explicitly excluded in #1).
3. One-off pages (e.g. "About", "My Projects")
4. Admin interface.
5. Multiuser

Of these, #1 is around 90% finished. #2 is in exploratory testing. #3 will take all of an hour. #4 and #5 are pipe-dreams for now.

Some of the first plugins I have in mind are:

1. Threaded comment system.
2. Gallery

Oh, and as you can see, the blog is named "Pentropy". I'll have a site/trac/svn up for it soonish.

I've had this idea for a while (it's been on the Breve to-do page as a speculative feature), but I also saw that Genshi supports such a feature and that there's a new template engine "Art" (not released yet) that will also have such a feature.

At first I thought of it as a cool, but not really important feature. I mean, you could do things like embed your last.fm playlist easily but other than that it seemed like fluff, so it was low on my priority list.

However, on the Pylons list, the author of Art mentioned how he planned to leverage this concept to easily support plugins. Duh! It's brilliantly simple (just how I like it). I was thinking in terms of pulling in XML data from other sites, but actually it could be used to pull a fragment from another controller on the same server. This would allow plugins to be nicely encapsulated and easily allow a page to be built from fragments generated by multiple controllers (something not currently possible with Pylons that I'm aware of), all from within the template.

I quickly hacked up an xinclude tag in Breve (mostly a thin wrapper around urllib2.urlopen() and it worked fine for pulling from remote sites. However, when I pointed it to another controller on the same Pylons app, things fell apart. The rendered template was incomplete. Further testing revealed that it only broke if the controller rendered another template (i.e. returning a simple string worked fine). My initial guess (without further testing) is that somehow the subrequest is happening withing the same context as the outer request or the template subsystem has an ugly global somewhere (i.e. the subrequest tries to reuse the existing Template instance). There's also the possibility that it's a bug in Breve, but this seems unlikely at this point (Breve is pretty simple).

My next phase is to narrow down where it's happening with the following tests:

1. Have a Pylons controller directly call another Pylons controller using urlopen and see what we get. Note that it must utilize the template subsystem since simple strings already appear to work.
2. If this works without issue, then work up a simple Breve app (outside of Pylons) and try to replicate the issue there.
3. If that works, then file a bug report on Pylons since I suspect it will be outside my abilities to track down.

Anyway, this feature seems so killer as to be well-worth the effort. I was casting about for ways to support plugins for this blog and this is clearly it.

Follow up:

Lo and behold, the bug was in Breve, and it was a pretty serious one. breve.Template was using register_flattener on a method named __slot which I had expected to not collide with other Template instances but it didn't work. What did work was making it an internal class, but what worked even better was avoiding register_flattener altogether and simply adding a __str__ method to the slot class. 1.0.32 is now the recommended release as it addresses this bug, a memory-leak issue (related to the first bug) and provides an initial (read: testing only) implementation of xinclude (although I think I'll rename it at some point).

The tag cloud on the right side is now a plugin. I utilized the new xinclude feature of Breve to create a component-based page.

The main blog controller:

class PostController ( BaseController ):

    # ...

    @jsonify
    api_tags ( self ):
        tag_index = Tag.select ( order_by = [ asc ( Tag.c._name ) ] )
        tags = [ ( t.name,
                   ( h.url_for ( controller='post', action='by_tag', id=t.id ),
                     len ( t.posts ) ) )
                 for t in tag_index ]
        return dict ( tags )

the plugin controller:

class TagcloudController ( BaseController ):
    def cloud ( self ):
        body = urlopen ( 'http://pentropy.twisty-industries.com/post/api_tags' ).read ( )
        c.tags = simplejson.loads ( body ).items ( )
        c.max = float ( max ( zip ( *zip ( *c.tags )[ 1 ] )[ 1 ] ) )
        return render_response ( 'tagcloud/cloud?fragment=1' )

the plugin's template:

div [
    ol ( class_ = 'tag-list' ) [ [
        li [
            span ( class_ = 'tag-context' ) [ '%d posts are tagged %s' % ( _posts, _tag ) ],

            a ( href = _link, class_ = 'tag',
                style = 'font-size: %0.2fem;' % ( 0.8 + _posts / c.max ) +
                        'color: rgb(%d,100,120);' % ( _posts / c.max * 180 ) +
                        'padding: %0.2fem;' % ( c.max / ( _posts or 1 ) / 10 ) )
            [ _tag ]

        ] for _tag, ( _link, _posts ) in c.tags
    ] ]
]

and finally, the index template looks like this:

# index.b
html [
    body [
        ...
        xinclude ( 'http://pentropy.twisty-industries.com/tagcloud/cloud' )
        ...
    ]
]

The basic sequence of events goes like this: a request is made for index.b (or rather a child template that inherits index, but that's not relevant here), index.b requests the plugin via the xinclude directive. The controller at the url passed to xinclude accesses the published API available in the main controller to get the JSON data describing the available tags (a list of ( tagname, ( link, article_count ) ) ). It then renders its own template (cloud.b) and returns the XHTML output back to the main template which injects it into its own final output.

It seems like a lot of steps for something that could have been (and originally was) simply an included template. However what it provides in return is the ability to define completely encapsulated plugins that have no knowledge of the internal workings of the main application. All they require is a defined JSON (or XMLRPC) interface to retrieve their data from.

The next step is to define a caching mechanism to reduce the number of times this complete sequence must be traversed. In the case of the tag cloud, it's clear we can cache until a new post is made (tags are never created independently of posts in Pentropy). To this end I'll probably add a generic cache directive that will be utilized something like this:

html [
    body [
        cache ( expires = c.tags_changed ) [
            h1 [ 'Tags' ],
            xinclude ( 'http://pentropy.twisty-industries.com/tagcloud/cloud' )
        ]
    ]
]

Having a generic cache directive will allow for static HTML fragments to be stored based upon specific criteria. More on this later.

Today I stumbled across Spawning, a high-performance WSGI server written by Donovan Preston. It's based on eventlet, which is a coroutine-based server framework for Python.

Initial reports deploying a Django application behind it appear promising, so I decided to give it a try.

For Pylons, the configuration required only a couple of changes to the Paste .ini file:

[server:main]
use = egg:Spawning
num_processes = 2
host = 127.0.0.1
port = 8000

I've deployed it on both this blog and on the Breve site. I'm planning on getting a proper benchmarking environment up later tonight so I can compare it to Paste#http.