Re: New stackless proposal

Mark Hahn wrote:
> Christian Tismer wrote:
...

>>Lots of details here. What I need is the overall picture.
> 
> It was hard enough to describe the details for now.  Especially since the
> proposal is still changing.  If it settles down I might be able to do better
> docs.
> 
> You should have read the overall proposal before replying.  :-)

Yes. And maybe after sleeping. :-)

>>What does "running" mean?
> 
> It means that the start() method has been called and the function has not
> terminated.

This is a vague definition.
I'm seriously asking what "running" should mean.
If something i waiting for an event, for instance,
it is suspended, not terminating. Is it running, then?

["you are more into threads than I am"]

> A thread is just a function that runs.  What could be simpler than that?   I
> think you are maybe assuming things about my threads that aren't there.

Maybe. It is probably the thread buzzword.

> When you pointed me to greenlets before all I saw was
> implementation-specific stuff for the Python stack.  I don't know how to
> translate that.

That was just a detail, not what I meant.
The Greenlets are somewhat a minimum implementation
of something switchable. There is no concept of
a thread, no pausing, no timeout, no priority,
no nothing. There is just a function to jump
off from one Greenlet and to continue with another
one. Every other concept can be put on top of that.
I'm trying to understand whether this is what you
are after, or if it is more.

[snipped agreed stuff]

>>I understand the equivalence with channels and their
>>flags for scheduling decisions.
>>Question: Who will do the scheduling?
> 
> I guess we'll have a fixed single scheduler object that can be overridden.
> You tell me.

Heh, this is a concpt that I'm seeking myself :-)

[again...]

>>Why should yield() be any different from yield(value) ?
> 
> So your channel.transfer() works without a value also?  If so then I'll make
> yield() work just like yield(value).  I thought a value was always needed.

For historical reasons, send passes a value and receive
returns a value. Internally, they do an exchange.
There is no reason not to echange two values, or
not to just do noting.
It is convenient to do a data echange, but it could
even be emulated. What is really needed is the
control flow.

[minor syntax stuff...]

>>Overloading, of course. What you want to express is a transfer()
>>from Modula II. I think it is fair to use this name.
> 
> Yes, transfer would be good if we don't go with yield.

[anonymous transfer]

>>I'm not sure if this is what you want. The explicit spelling
>>of a target thread is exactly what the Limbo people tried to
>>explictly avoid. The reason to have a channel at all is about
>>not having to name the other thread!
>>So I'd prefer to leave the rendevouz points as anonymous
>>as possible. This can be controlled by direct argument passing
>>easily.
> 
> Where would I pass this argument to?

I just have channel.send(value) and ret = channel.receive().
The partners don't need to know each other.

...

>>And what should happen after the Thread.kill() has surpassed the
>>timeout? Will you give up, then?
>>I have a much simpler strategy:
>>Whenever I have to kill a thread, I'm sending TaskletKill
>>exceptions until infinitum. This is no worse than current
>>Python behaviro: If a thread does not respond to certain
>>exceptions, there is nothing that can be done.
> 
> So you send the kill exception right away?  That is unsafe.  You cannot send
> a kill exception when the thread is in the middle of protected code or some
> resource could be left locked forever.  That is why I have the lock-block
> and wait to send the kill until the code is outside of the block. This is
> how ADA works.

Sure I can, it is not unsafe.
my kill exception is an ordinary Python exception that
is a subclass of SystemExit, namely TaskletExit.
Protected code is in try..finally blocks, and code
can always capture any exception.
The only unsafetyness is that my code might run
forever if it ignores exception. :-)

>  I don't care what happens on a time-out.  Once a time-out happens we could
> send the time-out exception to the kill sender and safely undo the kill
> attempt, if we wish, or we could still send the kill if it ever final gets
> out of the lock-block.  We do need to keep running the thread as long as it
> is in the lock-block so it can unlock the resource it is waiting on.
> 
> 
>>>Existing OS thread methods (*15)
>>>
>>>The existing Prothon Thread methods sleep(), running?(),
>>>and join() will be implemented on both new thread types.
>>>Logic will be implemented to make the different thread
>>>types compatible with each other when using these methods
>>>on Thread. For example when calling Thread.sleep(),
>>>if all Threads are idle then an OSThread.sleep() will be called.
>>
>>Sounds doable, but as said, I'd prefer to see a simpler
>>model, which is not trying to mimick the real thread
>>paradigm, but less.
> 
> 
> We have to have these methods for the OS threads.  I see no reason to not
> have them for the other threads also.  It is no effort to implement and it
> doesn't harm their functionality.  Correct me if I'm wrong.

> Also I really don't understand this simpler model.  Can you please try to
> explain what can be simpler?

Wel, I agree that you will want these functions most probably,
they just have the drawback that they force you into
decisions, early. If your thread can sleep, then
you need to decide whom to run, instead, who wakes it
up again, and so on.
By simple model I mean something that doesn't need this,
it is all up to the user. That's the basic part.
Everything that has to do with process control is
on a different layer.

[consumer/producer, who is on the stack]

> My proposal has the implementation that explains the answer to your
> questions.  The first next() call creates and calls the generator thread and
> then calls yield to get the first value.  Niether thread will have to
> reconstruct any stack state.  Both threads will have their own stacks
> existing the whole time they run.  The generator stack will run until it
> terminates and then it's stack will be destroyed.  The calling stack will be
> the normal stack that exists before and after the gen is used.

Ah, oh! I did not expect that you are doing assembly tricks.
Fine! Where do you draw the fence of this? If I want a
million of generators, will you allocate a million stacks?
(and be busy swapping instead of any computation? :-)

>>For my own current optimizations, I have the slight impression
>>that it makes things cheaper to have the iterator stay on
>>the stack, and the generator is to be re-run all the time.
>>Although I thing that real efficiency can only be done with
>>two real stacks and a real switch.
> 
> My proposal DOES have two stacks and a real switch.  Each thread has its own
> stack and the threadMgr is a real switch.  What can I do to explain my
> implementation better?

I completely understood.
Fine, I feared Stackless had a real competitor,
but this is not the case. :-))

I was talking of a cooperative interpreter implementation.
But if I see it reight, you dpn't make any use of it?
Switching stacks is really easy, but won't let you get
below thread sizes of Windows Fibers (64k I think).

> Or by real stack do you mean an OS stack?  I thought we were implementing
> stackless here.
> 
> By my count I have three things to change based on your suggestions above
> and several maybes.  You are asking for something simpler than threads but I
> don't understand what that something would be.  I think maybe this is just a
> matter of semantics.  You are suggesting we use a Python-type generator
> which Prothon already has and I'm suggesting we wait and see which works
> better.
> 
> I will implement the proposal on a trial basis with the method "transfer()"
> instead of "yield()" so it doesn't conflict with my "yield" keyword already
> implemented.  This will actually match what you want for generator.
> 
> So we are actually in pretty good agreement on all this.

Well, more or less, until I realized that you are
maybe not stackless at all, or you don't use it.
Funny :-)
Still I had no time to look into this.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:[email protected]>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/