Re: [pysqlite] Concurrency - first steps

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hugh Gibson wrote:
> And GIL means that working with the dict is threadsafe? 

Correct.

> In my own
> programming I always put a lock around that sort of access - maybe I
> don't need to.

In Python code you don't.  In C/C++/Java code you do.

The way thread safety is done in C/C++/Java is that by default anything
could be done by any other thread at any time.  So you have to carefully
go through your code ensuring that every line would still work correctly
despite any other line in the program running at the same time.  You
also have to apply the principle to any libraries you use.  The general
approach to spray locks throughout the code and hope you got everywhere
(something that is extremely unlikely).

Python's C code is the other way around due to the GIL.  When code is
running, no other code can run because the GIL is held.  Then in
carefully chosen places the GIL is released for sections of code that
are known to be safe while other threads are running, typically things
like I/O operations

As an example, if you had a hash table (dict) written in C/C++ without
any form of locks then you are 100% certain to have the fundamental
structure of the hash table trashed when using multiple threads.  As
Python does not release the GIL between byte code instructions, you
cannot trash a dict's underlying structure.

In all cases you are still subject to race conditions.  For example if
you do this:

  if "xyz" not in thedict:
     thedict["xyz"]=0
  thedict["xyz"]=thedict["xyz"]+1

No matter how well synchronized/locked the hashtable/dict is, the above
has race conditions.  However if the key ("xyz" in this case) were a
unique id such as the thread id then there is no race condition in
Python since no other thread has the same id.

Retrieving a value from a Python dict is always ok and there is no race
condition or possibility of a crash.  Doing so from a non-locked C/C++
style structure has all sorts of problems including important memory
referencing design issues.

I really like the way Python does threading under the hood.  Code is not
run concurrently unless you explicitly release the GIL for sections you
have designed/verified to be safe.  Other platforms require all code to
be 100% concurrency safe.

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkkjgrsACgkQmOOfHg372QRTrwCfWEu47d8IVDzAGtBYOMbgVgYF
Ot0AmQEa8iNFBvZxXZP0zVzo1VG+gGc+
=oJeh
-----END PGP SIGNATURE-----