async I/O via threads is extremly slow (Was: Does Python Need Virtual Threads?)

Hi,

async I/O in Python is extremly disappointing
and an annoying bottleneck.

The problem is async I/O via threads is currently
extremly slow. I use a custom async I/O file property
predicate. It doesn't need to be async for file

system access. But by some historical circumstances
I made it async since the same file property routine
might also do a http HEAD request. But what I was

testing and comparing was a simple file system access
inside a wrapped thread, that is async awaited.
Such a thread is called for a couple of directory

entries to check a directory tree whether updates
are need. Here some measurement doing this simple
involving some little async I/O:

node.js: 10 ms (usual Promises and stuff)
JDK 24: 50 ms (using Threads, not yet VirtualThreads)
pypy: 2000 ms

So currently PyPy is 200x times slower than node.js
when it comes to async I/O. No files were read or
written in the test case, only "mtime" was read,

via this Python line:

stats = await asyncio.to_thread(os.stat, url)

Bye

Mild Shock schrieb:
> 
> Concerning virtual threads the only problem
> with Java I have is, that JDK 17 doesn't have them.
> And some linux distributions are stuck with JDK 17.
> 
> Otherwise its not an idea that belongs solely
> to Java, I think golang pioniered them with their
> goroutines. I am planning to use them more heavily
> 
> when they become more widely available, and I don't
> see any principle objection that Python wouldn't
> have them as well. It would make async I/O based
> 
> on async waithing for a thread maybe more lightweight.
> But this would be only important if you have a high
> number of tasks.
> 
> Lawrence D'Oliveiro schrieb:
>> Short answer: no.
>>
>> <https://discuss.python.org/t/add-virtual-threads-to-python/91403>
>>
>> Firstly, anybody appealing to Java as an example of how to design a
>> programming language should immediately be sending your bullshit detector
>> into the yellow zone.
>>
>> Secondly, the link to a critique of JavaScript that dates from 2015, from
>> before the language acquired its async/await constructs, should be 
>> another
>> warning sign.
>>
>> Looking at that Java spec, a “virtual thread” is just another name for
>> “stackful coroutine”. Because that’s what you get when you take away
>> implicit thread preemption and substitute explicit preemption instead.
>>
>> The continuation concept is useful in its own right. Why not concentrate
>> on implementing that as a new primitive instead?
>>
> 
-- 
https://mail.python.org/mailman3//lists/python-list.python.org