Re: Help with external socket handling and xcb

Hi,

my mail program just informed me that there are new mails in my "xcb"
folder. No idea why it needed ten days to do so. Sorry!

Uhm. There is a lot going on in your mail. I'll skip all the "I wish
things where like XYZ", because I don't really know what to answer to
that. Sorry, but things are not like that currently.

Am 09.09.21 um 12:48 schrieb Carlo Wood:
[...]
>>> So, since it doesn't do that (as far as I know), it will have to
>>> make sure that every time it has something to write, it writes
>>> ALL of it before returning to my code... and that sounds very
>>> blocking. How can this ever work reliable with a non-blocking
>>> fd? Or is xcb just calling write in a loop until it wrote everything
>>> it has to write (also when write returns EAGAIN)?  
>>
>> Basically yes. _xcb_conn_wait() calls write_vec() to write some data
>> whenever poll()/select() say that the FD is writeable and there is
>> some pending data to send. Here is the loop that then calls
>> _xcb_conn_wait() until everything was written:
>> https://github.com/freedesktop/xcb-libxcb/blob/ee9dfc9a7658e7fe75d27483bb5ed1ba4d1e2c86/src/xcb_out.c#L457-L458
> 
> Just to be clear; so this means that in order for xcb_poll_for_event
> to work while only calling it when I see that the fd is readable, I only
> ever have to monitor that the fd is readable, and never have to worry
> about monitoring if the fd is writable, because all writing is done
> blocking, even though the fd itself is non-blocking?

Yup. You just monitor the FD for readability and then you can use
xcb_poll_for_event() / xcb_poll_for_reply() to check for "what is going
on". There is also xcb_poll_for_event_queued() which only checks the
internal buffer, but does not try to read anything from the socket.

But beware: Any XCB function can read from the socket. Even sending a
request may read from the socket (this is necessary to avoid a deadlock
with the X11 server). Thus, you should "pretend" that the socket is
readable after you wrote anything and call xcb_poll_for_event() /
xcb_poll_for_reply() afterwards.

> 
>>
>>> If anyone can give me pointers, or maybe tell me about a part of
>>> the API that I missed that is suitable for this, that would be
>>> great!  
>>
>> For non-blockingly waiting for a reply to a request: Do you know about
>> xcb_poll_for_reply()? There is an example on how to use that to get a
>> callback-based "your reply arrived"-API here:
>> https://gitlab.freedesktop.org/xorg/lib/libxcb/-/issues/52#note_712068
>> (Hm, it seems like this code does not handle sequence number
>> overflows...)
> 
> Hmm. This is still polling in nature and hard to wrap my head around
> without knowing more about the protocol that is being sent to and from
> the X server.

If you want to know more about the protocol and want to know how to
identify individual packages:

- Stuff being sent to the server has a length field in bytes 3/4 of the
packet. This is the number of four-byte-units of the packet.
- There is an extension for "big requests". In this case, the package
length will be zero and the following four bytes form a 32 bit length field.

- Stuff being received from the server always has at least 32 bytes.
- The first byte of a packet is its response_type
- Errors have response_type 0 and are always 32 bytes large
- Replies have response_type 1 and xcb_generic_reply_t describes their
layout. The length field is the
number-of-four-byte-quantities-after-the-32-bytes. So, a reply has
length 32 + 4*((xcb_generic_reply_t*)r)->length
- Events always have 32 bytes
- Except that this turned out not to be enough and the X11 generic
events extension was born. These events have response_type=35 and again
contain a length field (see xcb_ge_generic_event_t)

See here:
https://github.com/freedesktop/xcb-libxcb/blob/ee9dfc9a7658e7fe75d27483bb5ed1ba4d1e2c86/src/xcb_in.c#L236-L254

> Really non-polling would be when a callback for arrived replies is done
> whenever the function that reads incoming data (called by my
> application when there is data to read on the fd) sees that such
> replies did arrive.

Okay, then when the socket is readable, you do something like
dispatch_replies(conn, 1) from the code here:
https://gitlab.freedesktop.org/xorg/lib/libxcb/-/issues/52#note_712068
Additionally, you use xcb_poll_for_event() to check if an event was
received.

> I could call "poll_for_reply" whenever the socket becomes readable, but
> that makes no sense because at that moment the socket hasn't been read
> yet.

Both xcb_poll_for_event() and xcb_poll_for_reply() will try to read from
the socket when:

- there is no pending event/reply in the buffer and
- no other thread is currently trying to read from the socket

xcb_poll_for_queued_event() skips the "try to read from the
socket"-part. It was added because this unconditional trying to read
from the socket caused slowdowns with libX11. There is no
xcb_poll_for_queued_reply() currently.

Cheers & again sorry for the slow answer,
Uli
-- 
- He wants the impossible...!
- That's the short definition of 'captain'.