[ANN] HTMLTemplate 1.0.0

[has]

I got the impression that your HTMLTemplate was quite similar in concept to
TAL but less powerful (probably deliberately). Certainly it looks as though
most of your constructs map straight onto a very similar TAL expression.

Appearances, as they say, can be deceiving.<g> While some of the
original inspiration for HTMLTemplate did come from studying Zope Page
Templates (particularly the trick of hiding stuff in tag attributes so
templates can double as decent page mockups) the two systems are
_very_ different in concept and operation:

- TAL is based on a custom embedded language that provides basic
insertion and flow-control capabilities, allowing TAL templates to
pull data from a supplied data source to do their funky page rendering
thing.

- HTMLTemplate (like PyMeld and Nevow.Renderer) presents a template as
a very simple, custom DOM that is built and manipulated completely in
Python, the data being pushed into the compiled Template's object
model via a simple programmatic API.


Admittedly, HTMLTemplate's "compiler directives" do look a lot like
TAL expressions, at least to a casual observer, but their similarity
is superficial:

- TAL directives are responsible for controlling both template
construction AND rendering. The only thing your Python scripts do is
assemble your source data into TAL-digestible form, then serve it up
to the TAL template which will produce the rendered page.

- an HTMLTemplate directive is just a simple type declaration and name
binding - data used only to construct the template object mode, _not_
to render it. ALL presentation logic is implemented as part of your
Python scripts, which then interact directly with the Template object
model to produce the rendered page.


As to HTMLTemplate appearing "less powerful" than TAL, I suspect this
is more a problem of perception than actual implementation. (If
anything, would say the opposite was actually true, and HTMLTemplate
is probably a good bit more powerful than most other templating
systems available.) It's is certainly simple; however, that's not the
same thing as "low powered", and in this case Less is _definitely_
More. HTMLTemplate simply doesn't need to provide scads of built-in
functionality because it's designed from scratch to integrate
seamlessly with Python - allowing you to lever all the power of the
Python language and its myriad libraries at a level where most
templating engines can't/won't/don't even let you in. Something that
seems obvious to me now; but for a very long time (read: several
years) it wasn't.[1] Ah well, ya live and learn.<g>


Hope that clarifies (and if it tempts you to try it, maybe you can let
us know how that goes)

has

[1] It's no coincidence I started HTMLTemplate about the same times I
discovered Unix philosophy; i.e. "Keep it small, simple and highly
focussed, and design it to play really, really well with others." Best
advice ever. Took me three bloated, incomprehensible and appallingly
ill-conceived monsters to learn that lesson myself, but fourth time's
the charm, eh?

 

[David Bolen]

Now as to other template systems that are event driven system friendly,
the only one I know of for Python is Nevow, but that is bound up with
Twisted and thus couldn't be used with any other system. For my mind,
from the little I have looked at the Nevow examples, I am also not keen
on its magic system for callbacks dependent on naming. This sorts of
binds it quite tightly to a particular way of building your classes which I
would rather avoid.

Note that even if the template system isn't directly event driven
friendly, it can generally be used just fine with Twisted since you
can acquire your data prior to rendering the template. For example,
we are currently using ZPT with Twisted. We pre-compile the templates
for pages when the web objects (which are basic twisted.web Resource
objects) are built to establish the web site. Then, when asked to
render a page, we can take our time working up the necessary context
data (since the Twisted Resource object can return a pending indicator
and the object can then use the normal Twisted deferred processing to
complete the rendering). It's only once the data is fully realized
that we ask ZPT to use the compiled template to construct the HTML.
That operation is synchronous but there's no business logic running at
that point, and no timing dependency between the original request and
the final rendering.

HTMLTemplate could be inserted into the same mix as long as we held
off asking HTMLTemplate to render the DOM into HTML until we had all
the necessary data acquired to answer the various callbacks
immediately. That's probably the biggest advantage of Woven/Nevow in
a Twisted context with respect to a more MVC model - they integrate
cleanly into Twisted's event driven fabric with deferreds even during
individual element callbacks so you don't have to prepare all the data
in advance but can be driven from the template rendering process.

-- David

 

[has]

Now as to other template systems that are event driven system friendly,
the only one I know of for Python is Nevow, [...]

Note that even if the template system isn't directly event driven
friendly, it can generally be used just fine with Twisted since you
can acquire your data prior to rendering the template.

I think what Graham's getting at is that developers shouldn't have to
jump through such extra hoops just to compensate for design
deficiencies in the templating engine. The templating system ought to
integrate nicely with the rest of their system, not vice-versa.
HTMLTemplate is better than average here, but obviously still falls
short; we've been discussing off-list how best to rectify this
shortcoming, so will hopefully have something better soon.

HTMLTemplate could be inserted into the same mix

Ah, the damnation of faint praise... )

as long as we held
off asking HTMLTemplate to render the DOM into HTML until we had all
the necessary data acquired to answer the various callbacks
immediately.

Current thinking for the revised HTMLTemplate behaviour is:

1. Lose the main Template callback, whose only real raison-d'etre is
to automate the clone() and result() operations on either side of the
user's DOM manipulation. Eliminating this will better decouple data
insertion from HTML rendering at the cost of a couple more lines of
user code.

2. Modify the Repeater class's repeat() method, which currently
renders a list of items, replacing any previously rendered content
(meaning you have to have your whole list prepped in advance), to
allow it to be called any number of times, each time adding new items
to those already there (allowing items to be added as and when the
data becomes available).

That's probably the biggest advantage of Woven/Nevow in
a Twisted context with respect to a more MVC model - they integrate
cleanly into Twisted's event driven fabric with deferreds even during
individual element callbacks so you don't have to prepare all the data
in advance but can be driven from the template rendering process.

This is the level of flexibility I'd like to achieve with the revised
[event-friendly] HTMLTemplate design (though without losing the clean.
simple implementation and execution model it currently has). If you've
any suggestions here, please fire away.

 

[Chris Green]

It looks cool because it doesn't embed Python code in the template.
Do any of the other frameworks have this approach?

My favorite right now seems to be clearsilver ( www.clearsilver.net
). It has just enough functionality in the templates to allow people
to do a few presentation layer if statements. The key concept they
push is the backend populates a nested dataset that can be serialized
so designers and programmers can agree on the data format from the
start and go about their separate ways. Their website has a few
really good tutorials on how to use it. A really big plus for me is
since the do that it's a C library on the backend so if for whatever
reason you needed to reimplement the backend in something other than
python (god forbid ), you have that flexibility.

I tried to wrap my head around nevow and I decided I don't like
pulling data at the point at which we need it and I don't like coding
renderers in python. That really just makes me think at some point a
designer would be coming over asking to change the way something looks
by editing the rendering code.

The thing that I found coolest about nevow from a templating system
perspective was that it's templates could have renderable data in
them so the designer could have a template with real data in it. I
still have a sneaking suspicion that I'm missing the forest with
nevow however..

Where do people prefer to put the logic that says "this row should be
blue and the next one is white"?

 

[has]

My favorite right now seems to be clearsilver (www.clearsilver.net).

The key word here would be 'code', not 'Python'. ClearSilver does
embed code in its templates, so doesn't count here. (Sorry. Ditto
for PSP, TAL, Cheetah, EmPy, htmltmpl and various others I can't mind
offhand. To answer the OP's question, PyMeld, HTMLTemplate and
Nevow.Renderer keep all code completely separate of markup (there may
be others too, but I'm not aware of them myself).

It has just enough functionality in the templates to allow people
to do a few presentation layer if statements.

Personally, I prefer to avoid these kinds of systems wherever possible
as they tend to be large, complex, expensive to implement and extend,
and hard to do well. (c.f. Greenspun's Tenth Rule of Programming: "Any
sufficiently complicated C or Fortran program contains an ad-hoc,
informally-specified bug-ridden slow implementation of half of Common
Lisp."

There are some advantages to creating custom embedded templating
languages:

- Simple embedded languages are a good option for shrinkwrap software
aimed at non-programmers who may want to customise their templates a
bit, but don't want to learn a full-blown programming language to do
so.

- They can be implemented as efficient code generators or using
lower-level languages, making it easier to squeeze out maximum
performance. (Though bear in mind that templating engine efficiency
only makes sense when considered as a proportion of your entire
system's performance. If there's a lot of heavy-duty data-crunching
being done in the backend, say, the speed of the templating engine is
unlikely to have much impact overall.)


For example, Cheetah is maybe 2-3x faster than the latest (unreleased)
version of HTMLTemplate. Not really surprising: Cheetah's optimised
code generator beats out HTMLTemplate's DOM-style object-thrash for
raw rendering speed.

However, HTMLTemplate's codebase, API and documentation are maybe a
tenth the size of Cheetah's, making it much easier to learn and
modify. (It also made it much easier to write.

....

The key concept they
push is the backend populates a nested dataset that can be serialized

This approach is not unique to ClearSilver, or even embedded
language-based templating systems (e.g. see Ruby's Amrita system). I'm
not really a fan myself; the extremely narrow API leaves you no choice
about where and how you prepare your data, and having to transform
your data twice - converting from original format into intermediate
form and then into HTML - seems like extra work. There might be
situations where such separation is useful, but I wouldn't want it
forced on me - HTMLTemplate, etc. let the user choose.

so designers and programmers can agree on the data format from the
start and go about their separate ways.

This assumes that

 designers are also coders, with
responsibility for implementing and maintaining the presentation logic
that drives their templates. Some are, but others aren't. Again, I
don't think it's right for templating engines to _force_ programming
duties onto the designer. HTMLTemplate et-al let you choose.

....
[QUOTE]
A really big plus for me is that it's a C library on the backend so if for whatever
reason you needed to reimplement the backend in something other than
python (god forbid :)), you have that flexibility.[/QUOTE]

I'm still curious: how often does this actually happen? Or is it a
case of YAGNI?

(BTW, ClearSilver refuses to build on my OS10.2.6 system, so right now
it's -1 for me.:)

Anyway, I'd imagine if you're recoding, say, a multi-KLOC backend from
scratch, a few hundred more LOC spent porting the templating layer
probably isn't going to make much difference. One of the nice things
about HTMLTemplate is that its tiny size (350LOC) should make it very
easy to port too if your new language doesn't already have it.

....
[QUOTE]
I tried to wrap my head around nevow and I decided I don't like
pulling data at the point at which we need it and I don't like coding
renderers in python. [/QUOTE]

Can you be more specific about why you don't like it?

[QUOTE]
That really just makes me think at some point a
designer would be coming over asking to change the way something looks
by editing the rendering code.[/QUOTE]

This sounds like a demarcation issue:

- If the designer's job is purely to write HTML, then it's the
programmer's job to hook that HTML up to the presentation logic that
inserts content into it. Whether they like this arrangement or not is
irrelevant (see Golden Rule, The:). Keeping presentation logic
separate from HTML will make this process much less painful, however,
as it means the designer isn't stomping on code each time they make
HTML edits.

- If it's the designer's job to write their templates' presentation
logic as well, agreeing on a good API is no different to agreeing on a
good data exchange. A well-factored design should see presentation
logic in one set of files and business logic in another, allowing each
party to work on their own bit without bothering the other all the
time.

[QUOTE]
The thing that I found coolest about nevow from a templating system
perspective was that it's templates could have renderable data in
them so the designer could have a template with real data in it. [/QUOTE]

This is not unique to DOM-style templating systems (though they
certainly make it easy to do). e.g. See TAL, which embeds its custom
language in special tag attributes. The real benefit is in the
separation of markup and logic and what it does for the development
process; though the ability to use your finished templates as design
mockups is certainly nice.

[QUOTE]
I still have a sneaking suspicion that I'm missing the forest with
nevow however..[/QUOTE]

Hope this has helped a bit. :)

....
[QUOTE]
Where do people prefer to put the logic that says "this row should be
blue and the next one is white"?[/QUOTE]

I put it in the Template's Controller layer*, along with all the other
presentation control logic.

(* As-per Apple's definition of MVC, where 'Controller' is the layer
that connects the View to the Model.)


For example, given the following snippet of View code:

<table>
<tr node="rep:row" bgcolor="blue">...</tr>
</table>


It takes one line to assign the colour to the tag attribute, plus a
generic helper function (actually a generator), makeAlternator, that
yields each colour value in turn:

# Generic helper functions (usually put in a module of their own for
clarity and reuse):

def makeAlternator(*vals):
    i = 0
    while 1:
        yield vals[i % len(vals)]
        i += 1


# Template controller:

def renderTemplate(node, rows):
    node.row.repeat(renderRow, rows, makeAlternator('blue', 'white'))

def renderRow(node, row, alt):
    node.atts['bgcolor'] = alt.next()
    ...


# Output:

<table>
<tr node="rep:row" bgcolor="blue">...</tr>
<tr node="rep:row" bgcolor="white">...</tr>
<tr node="rep:row" bgcolor="blue">...</tr>
<tr node="rep:row" bgcolor="white">...</tr>
....
</table>