Re: How to monitor long-running fork in Skunkweb?

On Wed, Mar 09, 2005 at 08:18:10AM -0800, Drew Csillag wrote:
> I'm thinking that whether concurrency is a problem or not is
> dependant on the session store used.  It would seem that if you use a
> database backed session store, all is ok, the database handles the
> concurrency.  

Not really, at least not in a way that would prevent lost updates.
The session is loaded into memory in one transaction, and then it is
saved again in another, and the session is not locked for read/write
during the time it is loaded, only during the actual write operation.
 
> If you use the FSSessionStore, it's probably ok as long as the file
> writes are done a particular way.  I believe it's done correctly but
> haven't checked -- write to tempfile and rename, like the caching
> file writes are done such that you never see a half written file --
> if it's not done that way, it *should* be.  

Er, FsSessionStore doesn't do that, although it does use file locking.
The AE-based session store does, because it delegates to AE cache
calls.  Patching FSSessionStore to do use that technique for writes
would be a good idea for someone with write privileges to the SVN
repository.  (hint!)

> The only concurrency issues I can see with the session system in
> general is if you have multiple writers -- last one wins.  If you are
> going to have multiple writers, you can still mitigate the problem
> (though not remove it entirely) by doing your session changes
> quickly, i.e. refresh session data from the store (is there a method
> for this?), update [I'm thinking sess.update(changedict)], save; with
> no intervening delay.  Yes, you could still lose, but unless you are
> updating very frequently, the chances are minimal.  

Yup, this is the concurrency problem that I meant, and the chances are
actually a little worse than minimal.  I ran some calculations last
night based on these numbers:

  1. One process writing to the session every 10 seconds.
  2. Another process writing to the session every pi seconds.

If each session is open for 1/5 of a second, there will be 5
collisions in 1000 seconds.  Let's say that my estimate is an order of
magnitude too pessimistic -- that's still enough data loss that it
would happen every day.  

Most web applications aren't going to run into this problem, because
sessions aren't being written to concurrently except occasionally by
accident.

> If there was a method made [say, sess.atomic_update(changedict)], it
> could be a hook to the underlying session store to lock the
> underlying session resource while it does the refresh-update-save
> bit, in which case there shouldn't be any lossage.  You could still
> have a bit of lossage, *if* the multiple writers were updating the
> same *keys* in the session dict, but if that's an issue, *no*
> repository will probably help you with that.
> 
> But single writer, mulitple readers should always be ok right now, as
> the session data should *always* be consistent.  Actually, of ACID, I
> think we have A, C and if on a journalled FS, D.  Hey Jacob, not too
> shabby!  How's about we add a SQL interface?  <duck>
> 
> OTOH, I may be totally missing the whole point of the question, and
> be blabbering about incoherently....

No, this is helpful to discuss.  I think the sessionHandler api should
be extended so that you could get either a guaranteed latest copy of
the session, with write ability, or a read-only session that might be
concurrent but that you don't need to block to get.

> P.S.  Jacob, stop beating up the sessions!!!  Yes, they have a couple
> of corners and *theoretical* scalability problems, but in practice
> they really work well.

It is just a hang-up of mine.  I looked at some other session
implementations that did absolutely nothing to prevent concurrency
problems, so apparently people don't get as miffed about it as I do.

j
-- 
Jacob Smullyan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFCLzszuqamFyFXXLIRAhaAAKCEGhDx8oLG+zJsaH2+3eJRSKME3ACbBn9p
f8Hf2FoXrF6QHdQG1GkFw7U=
=P3dk
-----END PGP SIGNATURE-----