Re: Stdio.Buffer.set_buffer_mode revisited

Per Hedbor () @ Pike (-) importm?te f?r mailinglistan wrote:
> :-).  Well, it was because it originated from code that used a single
> socket to read/write.  I guess I could separate it now.

It would still be a single socket, just two explicit buffer objects.

It should simplify the code a bit, really. There is no real need to
have the two objects be aggregated in one, writing data to one is not
really related to reading from the other.

> I know.  Actually, the range_error() is the *only* kind of
> "conflict" which arises in this situation.  I tried looking for a
> way to only override range_error() in the read buffer version, but
> so far no luck.

There are actually some other possibilities, but it seems to be
handled correctly in the buffer.cmod code.

The main issue is that the buffer code tends to assume if the object
is the same so is the buffer (this would be an issue if you use this
'multi-buffer' object as input to write or the constructor of another
buffer as an example).


It also breaks the memory usage debug functionality, but I guess that
is not that critical. :)

>>  See also: All your wrappers for the various read_* functions.
> Is there anything amiss with those? I thought that was straightforward.

Well, yes, but they would not be needed at all if there was a separate
input and output buffer object.

> In order to reduce network traffic, reduce latency and increase
> throughput, I need to decide when to flush the network buffer (I end
> up with too many too small packets otherwise).  So the current
> version of the automatic output buffer flushing is unsuitable.  I'd
> love one where I can decide how much can be flushed.

Hm, well, that could be implemented. Currently it will flush when the
code returns to the backend (that is, much the same as when using
'normal' write callback based I/O).

>>o There is no need to have a write_cb, really, it should be
>>  pointless. Writing is handled by the file object in buffered mode.

> That's just it, I cannot use the buffered mode due to too small packaging
> on the network side.

This is somewhat surpising, but I assume it is because data is written
to the buffer from a different thread than the backend one?

In that case I can see why it could cause a lot of small writes, since
the backend is free to write the data you are just adding to the buffer.

However, is there any need to write the data from another thread in
the "normal" case?

> Well, in the application code I want to do things like:
>
>   c->read_int8() and c->read_hstring()
>
> and I want the call to block and wait.
>
> Yet, I want the input buffer to filled automatically from the
> network using buffer input mode.  The network socket is nonblocking by
> itself though.
>
> So, if you know a solution to this which avoid the Condition and the
> Mutex(es), I'd love to hear ideas.  But it seems to me that the functionality
> I need from Stdio.Buffer for the read side and for the write side
> needs a tad bit more native support to make it seamless.

In that case you would need the mutex+cond, yes.

Personally I would have written the code to be nonblocking at that
level, and do the blocking stuff at a level where it is easier
(in the query method, as an example).

-- 
Per Hedbor