Re: How to monitor long-running fork in Skunkweb?

--- Jacob Smullyan <[email protected]> wrote:

> On Wed, Mar 09, 2005 at 08:18:10AM -0800, Drew Csillag wrote:

> Not really, at least not in a way that would prevent lost updates.
> The session is loaded into memory in one transaction, and then it
> is
> saved again in another, and the session is not locked for
> read/write
> during the time it is loaded, only during the actual write
> operation.

I was coming from the single writer-many reader POV, in which it's
ok, but the multiple-writer issue is thornier...

> Er, FsSessionStore doesn't do that, although it does use file
> locking.
> The AE-based session store does, because it delegates to AE cache
> calls.  Patching FSSessionStore to do use that technique for writes
> would be a good idea for someone with write privileges to the SVN
> repository.  (hint!)

Ok, ok, I'll take the hint....  Probably sometime tonight or so.  The
file locking there shouldn't be problematic as is though.  I had
totally forgotten about the AE based session store...

> > The only concurrency issues I can see with the session system in
> > general is if you have multiple writers -- last one wins.  
...snip...
> > but unless you are
> > updating very frequently, the chances are minimal.  

> Yup, this is the concurrency problem that I meant, and the chances
> are actually a little worse than minimal.  I ran some calculations
> last night based on these numbers:
> 
>   1. One process writing to the session every 10 seconds.
>   2. Another process writing to the session every pi seconds.
> 
> If each session is open for 1/5 of a second, there will be 5
> collisions in 1000 seconds.  Let's say that my estimate is an order
> of
> magnitude too pessimistic -- that's still enough data loss that it
> would happen every day.  

Agreed, that's why I had suggested a way to refresh the session, just
before you make the change and save it back -- to minimize the time
window even more.  With non-synchronous file writes (remember the SMN
days w.r.t. FreeBSD and Slowlaris on this one?) on Linux, this should
be a very short time window indeed, on the order of a few
milliseconds (as opposed to 200ms).

> Most web applications aren't going to run into this problem,
> because
> sessions aren't being written to concurrently except occasionally
> by accident.

Mostly because people just can't click that fast, and if they do it's
usually the same form in which case:
   a) last one wins is what they want
   b) the same change would've been made in both cases anyway

> No, this is helpful to discuss.  I think the sessionHandler api
> should
> be extended so that you could get either a guaranteed latest copy
> of
> the session, with write ability, or a read-only session that might
> be
> concurrent but that you don't need to block to get.

I do like the idea of being able to get a latest & greatest copy of
the session, but I'm a little leery of a method to lock a session for
some indeterminate period of time, if only because I've seen some
ODBC drivers not bail on a SIGALRM and I suspect it's not just ODBC
drivers that have this problem, and if you held a locked session and
did a database op in the interim, one user could bollox up the server
very quickly.  I'd rather that it was an operation to say "change
these session keys to these values atomically, please".  This way the
lock lives just long enough to make the change, and gives the app no
real way to wedge the session. 

There is the whole problem of how an atomic update function would
deal with unsaved dirty session data at the time when this function
was called, which could be problematic (unless accounted for, of
course) for interations with session-using services (say, using
auth.SessionAuthBase or the like).  But perhaps any dirty keys could
additionally be written at the time of the atomic update.  We'd then,
of course need to track dirty keys, which is not necessarily bad.  It
would however, allow you to more efficiently store more than just a
pickle in the case of a database backed session store (I presume
that's how it works now).  You could then easily store it as a
3-tuple of sesssion-id, key, value in the DB, and then only changes
get written to the db, not the whole session dict.  Actually, in the
DB case, all session derefs could be lazy, i.e. fetch only when asked
for.  Then you could store all kinds of heinous stuff in a session
dict because it isn't all getting loaded on a session fetch.

Have we thought of (or implemented, I need to check) doing a Berkeley
DB session store?  Or even a BDB skunk cache (compile or component or
both)?  I think there may be some possibilities there.  Yeah I know,
using a database to not use a database... blah...blah...blah...  Or
have I turned into Ernest Chen?

Cheers,
Drew



	
		
__________________________________ 
Celebrate Yahoo!'s 10th Birthday! 
Yahoo! Netrospective: 100 Moments of the Web 
http://birthday.yahoo.com/netrospective/


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click