Re: mod_scgi on Apache2

Our 4-month old Chinese book/movie/music-based  social-network site  
douban.com is serving 150k pages (20k visits) daily. It's been  
running on apache2/mod_scgi/quixote/mysql on one single server from  
day one.

The scgi/quixote part of our site was fine, until we reached around  
50k hits per day. Then I started to have freezes occasionally. Now it  
freezes daily. When there're hundreds of users waiting, there's no  
time for any diagnosis. I just quickly clear up the old (scgi)  
processes and do a restart.

I've switched back and forth between scgi 1.4 and 1.2, and bumped up  
max_children to 10, and these made no difference.

Now the reason I've not reported this sooner. We're using memcached,  
to prepare for the (fingers crossed) arrival of the million page-view  
day. Most data queries go to memcache-python client first, then to  
mysql only if data is not found at the memcache. memcahed's python  
client is pretty much broken. besides data consistency, there've been  
reports of  crash/freeze.

I thought my problem was memcached related but changed my mind last  
week. The scgi freezes are rather random, but I did find one tiny  
pattern. It happened three times around the time mysql backup cron  
jobs run around 3 a.m. each day. mysqldump was configured to lock up  
each table in turn. so some of the unlucky write requests might have  
to wait for few minutes to get through. The backups took about 15  
minutes, but scgi froze right there, all the way to the morning when  
I woke up in panic.

I've considered downgrading to apache1, but saw this discussion and  
decided to be more constructive. Any hint will be appreciated.

A big thank-you to the quixote team, by the way.

- Bo

On Jul 19, 2005, at 2:40 AM, Neil Schemenauer wrote:

> On Mon, Jul 18, 2005 at 11:22:48AM -0700, [email protected] wrote:
>
>> What other kinds of problem reports have you gotten?  Things that
>> haven't been fixed?
>>
>
> The most recent report was of zombie Apache processes after the SCGI
> server had been down for a while.  I can't reproduce that one so I
> don't know what's going on there.  All the other problems that I
> know of have been fixed in the latest (unreleased) version.
>
> Obviously it would be nice to have someone like Jon Corbet of
> LWN.net say that mod_scgi for Apache 2 works for them.  Even without
> such feedback, I'm much more confident in module than I was when I
> first wrote that warning.
>
>   Neil
> _______________________________________________
> Quixote-users mailing list
> [email protected]
> http://mail.mems-exchange.org/mailman/listinfo/quixote-users
>