Re: no gil?

Hi,

if we look at Jython we already have a GIL less implementation of the 
Python programming language. I already had a project a few years ago 
that was only possible (to be exact: much simpler and therefore much 
cheaper) because I could utilize all 16 cores of a server using multiple 
threads.

IMHO the GIL is an implementation detail, that is almost invisible 
(except for performance, ctypes etc) for the Python developer. Therefore 
it is unrelated to the Stackless Python API. Of course we had to adapt 
the C-level implementation of Stackless.
But maybe we could base our tasklets on a more low-level construct, if 
we have to rework our implementation anyway. I remember Christians(?) 
mail about continuelets a few month ago.

Cheers
   Anselm


Am 17.04.2014 16:20, schrieb Kristján Valur Jónsson:
> Hi there.
> At PyCon I was approached by a core developer of CPython who asked me this:
> If we had a hypothetical GIL-less CPython, would stackless (tasklets or co-routines) still be a useful constructs or would real threads be sufficient?
>
> The question is serious and it is possible that a re-architecture of CPython will take place.  This would involve breaking API compatibility in some ways.  The question would be, if it would make sense for such a hypothetical Python to be fully non-recursive in its execution form, i.e. "stackless".  The way CPython uses the c-stack to mirror recursion on the Python stack is very convenient for the C API.
>
> So anyway, I couldn't answer that question right away and had to think it over.  Here are some of my thoughts so far:
>
> 1)       Scalability.  The reason that libraries like "gevent" exist, to provide a thread-like execution environment for concurrency.  You don't get added parallelism with gevent over regular threads but you get a program that scales better because OS threads are expensive.
>
> 2)      Picklability: In order to pickle (and restore) a python execution frame it must not be from a recursive invocation
>
> 3)      Co-operative scheduling:  To be able to run multiple execution contexts without pre-emptive switching is a huge win.  If your problem is such that you are IO bound anyway and cpu doesn"t matter much then this can be a very attractive programming model.
>
> Any other thoughts?  How would applications of stackless python be different if there were no GIL, i.e. threads could run in parallel (but tasklets were co-operatively scheduled in each thread as before)?  For one thing, I know that many synchronization privmitives currently written using channels would need re-writing.  They currently rely on tasklet.atomic for atomicitiy which also prevents thread switching (by preventing the thread from releasing the GIL).  If there were no GIL, we would need to use additional locking if we wanted our stackless locking primitives (e.g. stacklesslib.locks) to work between tasklets of different threads.
>
> Food for thought, anyway.
>
> K
>
>
>
> Hi there.
>
> At PyCon I was approached by a core developer of CPython who asked me this:
>
> If we had a hypothetical GIL-less CPython, would stackless (tasklets or
> co-routines) still be a useful constructs or would real threads be
> sufficient?
>
> The question is serious and it is possible that a re-architecture of
> CPython will take place.  This would involve breaking API compatibility
> in some ways.  The question would be, if it would make sense for such a
> hypothetical Python to be fully non-recursive in its execution form,
> i.e. “stackless”.  The way CPython uses the c-stack to mirror recursion
> on the Python stack is very convenient for the C API.
>
> So anyway, I couldn’t answer that question right away and had to think
> it over.  Here are some of my thoughts so far:
>
> 1) Scalability.  The reason that libraries like “gevent” exist, to
> provide a thread-like execution environment for concurrency.  You don’t
> get added parallelism with gevent over regular threads but you get a
> program that scales better because OS threads are expensive.
>
> 2)Picklability: In order to pickle (and restore) a python execution
> frame it must not be from a recursive invocation
>
> 3)Co-operative scheduling:  To be able to run multiple execution
> contexts without pre-emptive switching is a huge win.  If your problem
> is such that you are IO bound anyway and cpu doesn”t matter much then
> this can be a very attractive programming model.
>
> Any other thoughts?  How would applications of stackless python be
> different if there were no GIL, i.e. threads could run in parallel (but
> tasklets were co-operatively scheduled in each thread as before)?  For
> one thing, I know that many synchronization privmitives currently
> written using channels would need re-writing.  They currently rely on
> tasklet.atomic for atomicitiy which also prevents thread switching (by
> preventing the thread from releasing the GIL).  If there were no GIL, we
> would need to use additional locking if we wanted our stackless locking
> primitives (e.g. stacklesslib.locks) to work between tasklets of
> different threads.
>
> Food for thought, anyway.
>
> K
>
>
>
> _______________________________________________
> Stackless mailing list
> [email protected]
> http://www.stackless.com/mailman/listinfo/stackless
>

-- 
  Dipl. Phys. Anselm Kruis                       science + computing ag
  Senior Solution Architect                      Ingolstädter Str. 22
  email [email protected]             80807 München, Germany
  phone +49 89 356386 874  fax 737               www.science-computing.de
-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196