Re: Future directions for Cheetah (was Re: Tavis Rudd)

On Sat, May 02, 2009 at 04:56:55AM -0400, Tavis Rudd wrote:
> Tyler wrote:
> > Sounds good to me, we're pretty invested in Cheetah at Slide, and
> > there's definitely a *big* need to keep it current. Happy to hear you're
> > a nice reasonable chap ;) (I don't think we ever really exchanged emails
> > before).
> 
> Pleasure to meet you, Tyler. How are you using it at Slide?  Eric Huss, Max 
> Levchin.'s biz partner from NetMeridian (listbot), contributed a Cheetah 
> patch in 2007.  Is he involved in Slide?  Also I noticed Scott Banister on 
> Slide's board and I'm curious if you've got ex-IronPort 
> people on staff who had something to do with Cheetah's use at Slide.


Cheetah is used as our primary means of expressing 'views' inside our
web properties and applications. If you visit www.slide.com for example
you're seeing an Apache/mod_python instance (custom publisher) execute a
.pyc of a Cheetah template (we compile from .tmpl->.py->.pyc before
pushing production).

As far as Eric Huss, no clue who that is, so suffice to say he's not
that involved in Slide. We do have some ex-IronPort here at Slide, but
our engineering staff is a smidge ex-PayPal but mostly a new "original"
line up. The choice to use Cheetah was made long before I arrived, in mu
opnion, long before Cheetah was sane :-P (2.xx was a huge leap forward)


> In the past, Mike Orr has been a great 'house-keeper'.  Unfortunately, 
> there was no Mike around this time.  I should have sent a 'going away - 
> house keeper needed' post to the list.  However, please don't mistake my 
> going on vacation as a permanent abandonment of Cheetah or a sign that the 
> project was dead.  I took a six month vacation, and hiatus from Cheetah, in 
> 2004-05.  As Mike Orr pointed out, I've also disappeared for many shorter 
> periods over the years due to work pressures (custom CRM / ERP / 
> manufacturing workflow systems for startups on very tight schedules).  
> There was almost a 2 year gap between 0.9.16 and 0.9.17.  Scroll through 
> the CHANGES file or the cvs logs to see how much I contributed since that 
> hiatus (26 new releases)..


Interesting, I'm hoping my presence absolves some of that. I feel like
we're playing catch-up right now with other template engines at the
moment and the community is stuck in the mud. I'd like to see Google App
Engine adopt Cheetah, Django/Cheetah become more common, and maybe get
those Pylons folks to give it a whirl instead of Mako ;) I don't think
your absences are anything to really blame you for, shit happens, I
understand. Not speaking for Slide, but I'm of the opinion that we need
a good *well-supported* template system running our properties, and I
think Cheetah is it, but there's been numerous conversations along the
lines of "Cheetah's dead, maybe we should switch to
Mako/Breve/Genshi/Foo". 


> While I haven't, and don't plan to, abandon Cheetah I recognize that my 
> absences are alarming and annoying to patch contributors and newcomers with 
> questions.  It's not good for the community having me as the sole author 
> and maintainer.  I've been hoping that someone would step up to help 
> maintain it (coordinate patches, releases, documentation updates, question 
> answering, etc.) since Mike moved on.  Thank you for doing so!  


Ideally, I'd like to see "Cheetah core team" form, I'm fine with being
the de-facto maintainer and reviewer of patches, releases, etc, but a
good healthy project has usually between 3-10 "core members" who are all
familiar enough with the code base to assist with questions, review
patches, commit, etc

Not something I want to go through right now, but something to consider
moving forward (in mu opinion) as a necessity to grow the community and
developer base.


> > The import issues you're referring to were hidden behind a compiler
> > setting with v2.1.1. It's safe to assume v2.1.0.1 and v2.1.0 are
> > "broken" as far as backward compatibility is concerned.
> > 
> > Long story/email thread short, v2.1.1 "by default" works just like prior
> > versions of Cheetah with regards to the #import statement, so it being
> > the "official" release shouldn't be any concern IMHO.
> 
> Great. If all the tests are passing I'm satisfied.  I'm pleased to see that 
> you posted for user testing help on the list prior to the community 
> releases. 


In a blog post on the subject, I cited my desire to do things like this
due to being "scared to death" of breaking things :)

If you put yourself in my mind for a second, here I am a relative
unknown, offering up a community fork of Cheetah with barely any
street-cred to put forward, I was deathly afraid of losing the trust of
the community with a bunk release or two :)


> Regarding the import mechanism and support for conditional/fallback 
> imports, I think there's a better solution.  Rather than placing the 
> imports inline in method bodies, which is bad style IMO, we could add a 
> #module-header / #end module-header directive pair that accepts any 
> arbitrary cheetah directives (imports, flow control, try/except, etc) and 
> adds the generated Python code at the module level, just below where 
> Cheetah currently places import statements.  This has the advantages of 
> being more general/flexible, avoiding the hassle of needing multiple copies 
> of the try/import/except/import block if the imported symbols are needed in 
> several #def'ined methods, obviating the need for a compiler setting, and, 
> thus, being easier to explain.


While I agree it's not good style to perform inline imports, it's a
syntax that Python supports, and makes sense to me to support in Cheetah
as well. The canonical example I put forward in my commit messages was
not inline imports but rather blocks like the following:

	#try
		#import cjson
	#except ImportError
		#import simplejson
	#end try

We can discuss this further in another thread when you return however,
I'm up for a hearty language debate :D


> > How do you want to handle things moving forward? 
> .. 
> > Take a look at the roadmap I've outlined as far as the next couple
> > releases ... 
> > I'm anticipating v2.2 dropping around May 16th, and I *think* it should
> > be wholly backwards compatible back to v2.0.1. I've been monitoring
> > progress against the supported versions of Python with Hudson:
> >   http://hudson.communitycheetah.org
> 
> As 'maintainer' I think you'd be responsible for:
> - vetting patches and change suggestions with the help of me and others in 
> the community.  Changes to the language syntax and semantics must be 
> screened very carefully.  We have to guard against it becoming a grab-bag 
> of disjointed or inconsistent language constructs and features. Some have 
> argued that it already is (myself included).  Think of how careful Guido is 
> with changes to Python's syntax.  We want to emulate that.


Agreed, I'll be sure to put on my BFDL hat when screening language
changes ;)


> - ensuring that all new features and language constructs are accompanied by 
> test cases, good docstrings, and a good entry in the changelog (including 
> examples - see my changelog entries from the 2.0* releases). 


Test cases I agree, if you look back through some of my threads with
jbq@ I bugged him about test cases a couple times until we got some good
ones added into src/Tests/Unicode.py. I love me some PyUnit :)


> - ensuring that all changes preserve backwards compatibility as far as 
> possible: language features and semantics, the major API signatures, Python 
> version compat., etc. If incompatible changes are needed, they should be 
> highlighted in the release notes and CHANGES file.  We might also want to 
> tweak setup.py to display these notifications during the installion process 
> - Gentoo does something similar.


The toughest part about this is there's not a good list of "sample
templates" taken from the community that can be run against to verify
that we're still compatible with "their" setup. Particularly difficult
to test is compiled-template compatibility, i.e. a template compiled
against v2.0.1 will work properly against a v2.1.1 library. Any
suggestions on testing this in an automated fashion? (also see:
http://hudson.communitycheetah.org/)


> - helping decide when it is time to ditch old unused or bad features. I 
> feel that a Python 3.0 compatible release would be an opportune time to rip 
> out a few things from Cheetah's early days: all the WebWare related 
> 'transaction' stuff is at the top of my list.  I'd also love to axe the 
> #include directive as I view it as an ugly cross-over from PHP and there is 
> always a better way of doing anything done with #include.  Unfortunately, I 
> think too many people use #inherit so it'll have to stay.  Thanks to 
> compiler settings, such changes can be done in a backwards compatible way.  
> We'd simply leave the old behaviour/features enabled in the existing 
> Template class and provide a subclass, called something like Template3, 
> that has them disabled in its default compiler settings.  On a related 
> note, I think Cheetah should come with a Template subclass that has 
> everything related to NameMapper and the searchList disabled by default.  
> They are the most confusing features for newcomers and the most 
> misunderstood by critics.  I use such a subclass in most of my own projects 
> because of the performance advantages.


Let's talk more about this in another thread, that WebWare stuff I'd
LOVE to axe, like LOVE LOVE LOVE.

*chop chop chop*


> - checking accepted patches in (or simply merging)
> 
> - coordinating the coding of other changes, or coding them yourself  
> 
> - cutting releases, pgp signing them, and announcing them via SF, the 
> website, the mailing list and PyPi.  You might also have to nag downstream 
> packagers to stay current (I have Gentoo and their package keywords in 
> mind).


The Debian maintainer is MIA, yeah, you'll have to add my as a
contributor to the Cheetah pypi project since I cannot release through
there (I've been pushing under Community Cheetah)


> - coordinating the testing of rc releases.
> 
> - kickstarting and coordinating updates to the external documentation.  
> Also, my lame example on the website needs replacing with a several better 
> examples, including examples of the basic API usage patterns.


mobiledreamers put forth the idea of the "Cheetah Cookbook" which I
think would be a great way to put Cheetah examples on to the site.


> - updating the contributors list, the 'who uses Cheetah' page, and the list 
> of 3rd party articles and documentation on the website.

Yeah, removing reddit might be a good idea too since they don't use
Cheetah anymore :(

> - answering questions on the mailing list or nudging other users to 
> contribute answers ... not my strength ;)

No kidding ;D

> - coordinating a roadmap for future releases, as you've already done.  I've 
> resisted doing this in the past as I was usually the sole coder and it was 
> dangerous with my work schedule to commit to anything additional.


The fun thing about open source roadmaps is unless you're ubuntu or
Gnome, nobody *really* holds your feet to the fire ;) That said, I work
a lot and Cheetah is a definitely on the upper end of my priority list
so I think I can commit time to things I've put on the roadmap


> The roadmap looks good and mirrors some rough plans I jotted out last year. 
>  Python 3.0 compatibility should be prime goal over the next year, IMO.  
> Everything else should be a stepping stone towards that.  The use of 
> unicode everywhere internally is the most important one (Cheetah shows its 
> age here).  I don't see the point of analyzing the run-time performance of 
> templates till that goal has been reached.  Keep in mind that I and others 
> have already spent a great deal of time doing just that and Cheetah is 
> currently one of the fastest Python template engines if used sanely.  (The 
> parser is a bit inefficient but that's another matter and isn't worth 
> worrying about now.)


Note that I said "analyzing" not "fixing" :) What I intend on doing is
using statprof.py and setting up a set of "standard performance test"
that way I/you/we can run the standard performance tests against new
patches/releases to verify that we're getting faster, not slower. It
also might be interesting to see if there's any bottlenecks sitting
around.


> Can you expand on "'Modernize' other aspects of Cheetah infrastructure"?  I 
> had some plans that might overlap.

AXE WEBWARE! Moving all objects to new-style objects, slowly phasing out
the use of builtins that did/might go away in Python 3000, things like
that. These aren't concrete plans just yet, but I thnk you get the gist.


> One very important thing missing from the roadmap is an update of the 
> documentation.  Whether it's done with LaTex, Sphinx or something else, it 
> needs doing.  Mike's old 1.0 docs in LaTex are quite out of date now.  I 
> certainly don't have time to do it and, unfortunately, previous volunteers 
> never got far past debating what format to use.

Gahhhh documentation! :) I don't quite know LaTex (how I've survived as
a developer without learning it is a mystery) but I can't really think
of any other good format to put documentation in that can be "built" to
other formats (PDF, HTML, etc). SGML might be an option, there's also
some things that the Git project does with documentation that's a little
different. This is a good subject for another thread to start when you
return to Canada.


> Some random points:
> * cvs
> I agree, it rots the brain.  I haven't used it outside of Cheetah since 
> 2004 - Hg for everything else now.  I already had plans to switch Cheetah 
> to Hg, but let's make github the official repo.  Can you setup a clone on 
> github that's under the username 'cheetah', rather than rtyler, that we can 
> use for official branches? 

We could use SF.net's Git support? Fsck it, done:
	http://github.com/cheetahtemplate/cheetah/tree/master


> * mailing list
> This is one of the few things SF does a decent job of.  I see no reason to 
> change anything there.  I'll send you the list admin password.


I believe with SF.net admin rights, I have access to the list, I could
be wrong.


> * tarball file releases
> This should stay on SF for continuity and because downstream packagers 
> (such as Gentoo) rely on the files being there.  There's a script in the 
> Cheetah src that automates the packaging and upload for a new release.  
> You'll need GnuPG configured to sign the tarball.  Once the new files are 
> on SF you'll need to login, add a new file release with the relevant bits 
> from the CHANGES file as the release notes and then select and tag the 
> files.  It's a bit tedious so you might want to look into one of the SF 
> automation tools out there.


Agreed, but some of your scripts, at least in the tree are quite Tavis
specific :) We can discuss this later.


> * comments about old code / semantics
> I'd appreciate it if you didn't add unsigned comments like '(read: broken)' 
> to the changelog or 'this is stupid' to source code. The former is 
> debatable and for the latter look at the module docstring in 
> unittest_local_copy to see why it exists (keep your own comments on forking 
> in mind when you read it - I agree with them btw and maintain several 
> private forks).  If you disagree with something and want to add a 
> constructive comment about it please sign your comments.  


Heh, this is one of my bad habits ;) We recently did a grep of the Slide
codebase and I think I contributed the majority of the expletives. I'll
"sign" things with my handle "tyler@" instead of initials, that was
crazy confusing when I first started diving into this code.


> By the way, I am tempted to add 'this is stupid' comments to some of my own 
> code in Cheetah.  Most of it was written a long time ago and my style and 
> skills have changed a lot since then.


Understandable :)


> * changelog entries (the CHANGES file)
> They should be as detailed as possible and include examples where 
> applicable.  Please sign your own changelog entries.  Traditionally we used 
> the contributers initials in square brackets - see the very bottom of that 
> file for a key.  However, email sigs are fine as well.
> 
> * bug tracker
> Mantis is much better than the crap one on SF and is more accessible to the 
> community than the old bugs file.  Thanks.  We can setup the subdomain 
> bugs.cheetahtemplate.com when I'm back.


Agreed, I'm a big fan of Mantis, primarily for features like issue
relationships, and automatic changelog generation (based off of issues)

Sounds good


> * website
> I think, the existing website is fine, with the exception of examples.  
> I'll change the contributors page and footer to say that you are the 
> current maintaineer late next week.  Have you seen the script and template 
> I use to update the static pages?  Aside from bugs.ct.org can you think of 
> any other subdomains we should create.  We should add nav entries for 
> github and mantis and add the roadmap somewhere.

I've seen the script. Bummer my two domains (communitycheetah.org/.com)
will be going to waste now. Maybe I can organize a cat community..

bugs.cheetahtemplate.org
hudson.cheetahtemplate.org

If you don't mind, I'd like to break the site out of that uber-template
_SiteTemplate.tmpl so it's a bit more accessible to patches by others
and updating. If you refer to the gh-pages branch in Git
(http://github.com/rtyler/cheetah/tree/gh-pages) this is where the
communitycheetah.org site is stored, and it uses some "modern" Cheetah
features like Markdown and inheritance which IMHO makes it far easier to
work with.


> * irc
> I see that you've already got that handled.  Thanks.  My response to Juha 
> when he brought it up before was that he'd have to set it up as I don't 
> have time for irc.  I never hang out on irc and almost never on other types 
> of chat these days.

OMG SO NO AIM? LOLZ A/S/L?


> > I couldn't find recent traces of you online which
> > I think spooked some of the community and myself.
> 
> Understandably. Outside of Cheetah and internal work stuff, I keep a very 
> low profile online (no Facebook or Slide for me ;).  That will change over 
> the next year or so, as I have several other open-source licensed, but 
> unpublicized, projects that I'm hoping to release soon.
> 
> We should talk on the phone when I'm home.

That sounds like a good idea, when might this be around? If you've not
figured it out already, I'm about as patient as a little kid that has to
pee.

This summer I might just have to travel up to your neck of the woods
from San Francisco (Vancouver is /almost/ as beautiful as San Francisco
in the summer ;))

> p.s. Please excuse any odd typos in this.  I had to type it on my cellphone 
> and the font size is really small.

That's the most ridiculous thing I ever heard :-P, your thumbs are
surely going to blister up now.


Cheers
-- 
-R. Tyler Ballance
Slide, Inc.

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations 
Conference from O'Reilly Media. Velocity features a full day of 
expert-led, hands-on workshops and two days of sessions from industry 
leaders in dedicated Performance & Operations tracks. Use code vel09scf 
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf

_______________________________________________
Cheetahtemplate-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkn8xKgACgkQFCbH3D9R4W+yrQCcD3jQd9TH5Dj34S+WqRIxhv0E
H8IAniacC4eFx/XI3kjCP/1R/bdkjxLS
=ugRt
-----END PGP SIGNATURE-----