Re: Rebel With A Cause

On Friday, October 31, 2003, at 11:06 PM, Miles Egan wrote:

> Very nice writeup!  The lisp world could use more of these.  Maybe some
> room could be made on common-lisp.net to store such things?
>
> A few questions:
>
> 1. You mention Smalltalk.  Have you done much web development in
> Smalltalk?  How would you compare it to lisp for this kind of work?

Before Java came along, I did a lot of Smalltalk and I liked it a lot - 
the 'everything is an object' way of thinking and the integrated, 
interactive environment. But I didn't do any web application 
development work in Smalltalk, so I can't compare from experience.

I did a lot of Lisp during at the university and in my first job, 
before my Smalltalk years. After that came many years of Java - still 
the main thing we do in our company.

I revisited (Common) Lisp last year, I guess after reading some of Paul 
Graham's writings. And although I think I was pretty good at Lisp in 
the past, I must admit that revisiting everything after the 
Smalltalk/Java experiences, I now have a much deeper understanding 
about Lisp and some of the areas that remained vague in the past.

> 2. How do you make modifications to the site?  Do you connect to the
> live server and work in the repl or do you do a more conventional
> release cycle?

The whole project is in CVS, by RSYNCing the persistent data (the 
prevalence root and blobs) from the server to a local machine, the 
application can be run from any workstation.

Development and testing is then done locally. We then SSH into the 
server and do a CVS update. When the changes are only in HTML, image or 
LSP files, nothing more has to be done. When Lisp code has changed, we 
telnet into the Lisp Application Server to get a REPL and invoke an 
ASDF build, optionally doing some fiddling with live data structures. 
In most cases, users won't notice anything. It is like in Paul Graham's 
stories: they call to report a bug and a bit later we ask them to try 
again and it works (apart from the various caches and proxy servers 
between a web client and a web server ;-). In some cases (like when the 
definition itself of a web app changes - adding new LSP's or actions) 
the web app has to be restarted. This is so fast that I don't think 
users can notice it.

> 3. You mention that it took some time to figure out how to make the 
> best
> use of the prevalence layer.  Can you elaborate on this?  How do you
> handle running out of memory?

The out of memory (and related garbage collection) question comes up a 
lot. I think that as long as the size of your DB fits nicely in your 
Lisp image, there won't be much problems. I guess that on machines with 
1GB of RAM, your Lisp image can easily hold anything from 1Mb to 100Mb 
worth of data (OK, that last upper limit is just an educated guess). 
For databases in that range (and I think a large number of databases 
fit in this range), Prevalence does make a lot of sense. It is some 
much more productive (let alone faster during runtime).

Using Prevalence for the first time in a production environment was 
exciting. My main thought or concern was: let's hope that it will hold, 
that the data would remain consist and didn't get lost. That is why we 
added backup procedures. We never lost any data, but once we had a 
partial corruption of some inter-object relations (some references to 
what had to be the same object where actually to two identical copies 
of that object). By debugging and fixing this in a copy from the 
production environment and then running it on the live system we were 
able to fix it. Something like that is virtually impossible in Java.

> 4. Was your experience with PAServe positive?  Any gotchas?

I think (Portable) AllegroServe is absolutely great - it works as 
expected, is fast and as far as I know bugfree - there is no such thing 
as bugfree, but we just didn't encounter any. No gothas. I had to do a 
little ugly hack to add request attributes to the request object (just 
changing the class definition, instead of subclassing - a modification 
like that wasn't really forseen I guess, or I missed the clean way to 
do it.)

> Anyway, thanks for the detailed review.

You're welcome,

Sven