Future directions for Cheetah (was Re: Tavis Rudd)

Tyler wrote:
> Sounds good to me, we're pretty invested in Cheetah at Slide, and
> there's definitely a *big* need to keep it current. Happy to hear you're
> a nice reasonable chap ;) (I don't think we ever really exchanged emails
> before).

Pleasure to meet you, Tyler. How are you using it at Slide?  Eric Huss, Max 
Levchin.'s biz partner from NetMeridian (listbot), contributed a Cheetah 
patch in 2007.  Is he involved in Slide?  Also I noticed Scott Banister on 
Slide's board and I'm curious if you've got ex-IronPort 
people on staff who had something to do with Cheetah's use at Slide.

I've been on the road since late Nov - visiting my family in NZ all Dec 
then this trip down south with my girlfriend - and mostly offline since Jan 
2nd, just before your first posting to the list. I was so busy with work 
leading up to this trip, and even during the first month of it, that 
Cheetah was a very low priority, especially considering there hadn't been 
any activity on the list in almost two months and the three, reasonably 
minor, patches since the last release had been committed to the cvs repos. 
and were available to all who needed them.

In the past, Mike Orr has been a great 'house-keeper'.  Unfortunately, 
there was no Mike around this time.  I should have sent a 'going away - 
house keeper needed' post to the list.  However, please don't mistake my 
going on vacation as a permanent abandonment of Cheetah or a sign that the 
project was dead.  I took a six month vacation, and hiatus from Cheetah, in 
2004-05.  As Mike Orr pointed out, I've also disappeared for many shorter 
periods over the years due to work pressures (custom CRM / ERP / 
manufacturing workflow systems for startups on very tight schedules).  
There was almost a 2 year gap between 0.9.16 and 0.9.17.  Scroll through 
the CHANGES file or the cvs logs to see how much I contributed since that 
hiatus (26 new releases)..

While I haven't, and don't plan to, abandon Cheetah I recognize that my 
absences are alarming and annoying to patch contributors and newcomers with 
questions.  It's not good for the community having me as the sole author 
and maintainer.  I've been hoping that someone would step up to help 
maintain it (coordinate patches, releases, documentation updates, question 
answering, etc.) since Mike moved on.  Thank you for doing so!  

> The import issues you're referring to were hidden behind a compiler
> setting with v2.1.1. It's safe to assume v2.1.0.1 and v2.1.0 are
> "broken" as far as backward compatibility is concerned.
> 
> Long story/email thread short, v2.1.1 "by default" works just like prior
> versions of Cheetah with regards to the #import statement, so it being
> the "official" release shouldn't be any concern IMHO.

Great. If all the tests are passing I'm satisfied.  I'm pleased to see that 
you posted for user testing help on the list prior to the community 
releases. 

Regarding the import mechanism and support for conditional/fallback 
imports, I think there's a better solution.  Rather than placing the 
imports inline in method bodies, which is bad style IMO, we could add a 
#module-header / #end module-header directive pair that accepts any 
arbitrary cheetah directives (imports, flow control, try/except, etc) and 
adds the generated Python code at the module level, just below where 
Cheetah currently places import statements.  This has the advantages of 
being more general/flexible, avoiding the hassle of needing multiple copies 
of the try/import/except/import block if the imported symbols are needed in 
several #def'ined methods, obviating the need for a compiler setting, and, 
thus, being easier to explain.
 
> How do you want to handle things moving forward? 
.. 
> Take a look at the roadmap I've outlined as far as the next couple
> releases ... 
> I'm anticipating v2.2 dropping around May 16th, and I *think* it should
> be wholly backwards compatible back to v2.0.1. I've been monitoring
> progress against the supported versions of Python with Hudson:
>   http://hudson.communitycheetah.org

As 'maintainer' I think you'd be responsible for:
- vetting patches and change suggestions with the help of me and others in 
the community.  Changes to the language syntax and semantics must be 
screened very carefully.  We have to guard against it becoming a grab-bag 
of disjointed or inconsistent language constructs and features. Some have 
argued that it already is (myself included).  Think of how careful Guido is 
with changes to Python's syntax.  We want to emulate that.

- ensuring that all new features and language constructs are accompanied by 
test cases, good docstrings, and a good entry in the changelog (including 
examples - see my changelog entries from the 2.0* releases). 

- ensuring that all changes preserve backwards compatibility as far as 
possible: language features and semantics, the major API signatures, Python 
version compat., etc. If incompatible changes are needed, they should be 
highlighted in the release notes and CHANGES file.  We might also want to 
tweak setup.py to display these notifications during the installion process 
- Gentoo does something similar.

- helping decide when it is time to ditch old unused or bad features. I 
feel that a Python 3.0 compatible release would be an opportune time to rip 
out a few things from Cheetah's early days: all the WebWare related 
'transaction' stuff is at the top of my list.  I'd also love to axe the 
#include directive as I view it as an ugly cross-over from PHP and there is 
always a better way of doing anything done with #include.  Unfortunately, I 
think too many people use #inherit so it'll have to stay.  Thanks to 
compiler settings, such changes can be done in a backwards compatible way.  
We'd simply leave the old behaviour/features enabled in the existing 
Template class and provide a subclass, called something like Template3, 
that has them disabled in its default compiler settings.  On a related 
note, I think Cheetah should come with a Template subclass that has 
everything related to NameMapper and the searchList disabled by default.  
They are the most confusing features for newcomers and the most 
misunderstood by critics.  I use such a subclass in most of my own projects 
because of the performance advantages.

- checking accepted patches in (or simply merging)

- coordinating the coding of other changes, or coding them yourself  

- cutting releases, pgp signing them, and announcing them via SF, the 
website, the mailing list and PyPi.  You might also have to nag downstream 
packagers to stay current (I have Gentoo and their package keywords in 
mind).

- coordinating the testing of rc releases.

- kickstarting and coordinating updates to the external documentation.  
Also, my lame example on the website needs replacing with a several better 
examples, including examples of the basic API usage patterns.

- updating the contributors list, the 'who uses Cheetah' page, and the list 
of 3rd party articles and documentation on the website.

- answering questions on the mailing list or nudging other users to 
contribute answers ... not my strength ;)

- coordinating a roadmap for future releases, as you've already done.  I've 
resisted doing this in the past as I was usually the sole coder and it was 
dangerous with my work schedule to commit to anything additional.

The roadmap looks good and mirrors some rough plans I jotted out last year. 
 Python 3.0 compatibility should be prime goal over the next year, IMO.  
Everything else should be a stepping stone towards that.  The use of 
unicode everywhere internally is the most important one (Cheetah shows its 
age here).  I don't see the point of analyzing the run-time performance of 
templates till that goal has been reached.  Keep in mind that I and others 
have already spent a great deal of time doing just that and Cheetah is 
currently one of the fastest Python template engines if used sanely.  (The 
parser is a bit inefficient but that's another matter and isn't worth 
worrying about now.)

Can you expand on "'Modernize' other aspects of Cheetah infrastructure"?  I 
had some plans that might overlap.

One very important thing missing from the roadmap is an update of the 
documentation.  Whether it's done with LaTex, Sphinx or something else, it 
needs doing.  Mike's old 1.0 docs in LaTex are quite out of date now.  I 
certainly don't have time to do it and, unfortunately, previous volunteers 
never got far past debating what format to use.

Some random points:
* cvs
I agree, it rots the brain.  I haven't used it outside of Cheetah since 
2004 - Hg for everything else now.  I already had plans to switch Cheetah 
to Hg, but let's make github the official repo.  Can you setup a clone on 
github that's under the username 'cheetah', rather than rtyler, that we can 
use for official branches? 

* mailing list
This is one of the few things SF does a decent job of.  I see no reason to 
change anything there.  I'll send you the list admin password.

* tarball file releases
This should stay on SF for continuity and because downstream packagers 
(such as Gentoo) rely on the files being there.  There's a script in the 
Cheetah src that automates the packaging and upload for a new release.  
You'll need GnuPG configured to sign the tarball.  Once the new files are 
on SF you'll need to login, add a new file release with the relevant bits 
from the CHANGES file as the release notes and then select and tag the 
files.  It's a bit tedious so you might want to look into one of the SF 
automation tools out there.

* version numbering
In the past I've been fairly conservative with the release numbering: 
issuing alpha and beta releases for major changes (or major internal 
refactoring) followed up by 'rc' releases that are out for at least a week 
of user testing before the final release. For minor feature adds or bug 
fixes that aren't likely to break things, I've gone straight to the final 
release (e.g. 2.0.1).  I'll leave that judgement call up to you for 2.2. 
Cheetah version numbers are always 2 or 3 digits with an optional alpha, 
beta or rc flag.  I've never used 4 digits like 2.1.0.1.  See 
src/Version.py.

* comments about old code / semantics
I'd appreciate it if you didn't add unsigned comments like '(read: broken)' 
to the changelog or 'this is stupid' to source code. The former is 
debatable and for the latter look at the module docstring in 
unittest_local_copy to see why it exists (keep your own comments on forking 
in mind when you read it - I agree with them btw and maintain several 
private forks).  If you disagree with something and want to add a 
constructive comment about it please sign your comments.  

By the way, I am tempted to add 'this is stupid' comments to some of my own 
code in Cheetah.  Most of it was written a long time ago and my style and 
skills have changed a lot since then.

* changelog entries (the CHANGES file)
They should be as detailed as possible and include examples where 
applicable.  Please sign your own changelog entries.  Traditionally we used 
the contributers initials in square brackets - see the very bottom of that 
file for a key.  However, email sigs are fine as well.

* bug tracker
Mantis is much better than the crap one on SF and is more accessible to the 
community than the old bugs file.  Thanks.  We can setup the subdomain 
bugs.cheetahtemplate.com when I'm back.

* website
I think, the existing website is fine, with the exception of examples.  
I'll change the contributors page and footer to say that you are the 
current maintaineer late next week.  Have you seen the script and template 
I use to update the static pages?  Aside from bugs.ct.org can you think of 
any other subdomains we should create.  We should add nav entries for 
github and mantis and add the roadmap somewhere.

* irc
I see that you've already got that handled.  Thanks.  My response to Juha 
when he brought it up before was that he'd have to set it up as I don't 
have time for irc.  I never hang out on irc and almost never on other types 
of chat these days.

> I definitely want you to stay around since I think it's a good thing for
> a community to have it's Linus (ex: Junio is the maintainer of Git, but
> Linus (author) is still pretty active in the community).

Don't worry I'll be around on the mailing list again once I'm back in 
Vancouver.  I only disappear on long trips like this once every 4 or 5 
years.  I'll be paying close attention to backwards compat as I maintain 
several large codebases at work that rely on Cheetah working the way it 
does now.

> I couldn't find recent traces of you online which
> I think spooked some of the community and myself.

Understandably. Outside of Cheetah and internal work stuff, I keep a very 
low profile online (no Facebook or Slide for me ;).  That will change over 
the next year or so, as I have several other open-source licensed, but 
unpublicized, projects that I'm hoping to release soon.

We should talk on the phone when I'm home.

Fyi, we're heading out into the Bolivian backcountry tomorrow and I'll be 
completely offline till Wed night.

Thanks again for stepping up,
Tavis

p.s. Please excuse any odd typos in this.  I had to type it on my cellphone 
and the font size is really small.

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations 
Conference from O'Reilly Media. Velocity features a full day of 
expert-led, hands-on workshops and two days of sessions from industry 
leaders in dedicated Performance & Operations tracks. Use code vel09scf 
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf