Re: Updated dataflow semantics for RTT

--===============0193899577==
Content-Type: multipart/alternative; boundary=001a11c3288294673c05209c22d3

--001a11c3288294673c05209c22d3
Content-Type: text/plain; charset=UTF-8

>
> It doesn't fix the out-of-order bug for the buffer-per-channel case.
> You can still starve a certain channel and get old data much later.

That's why we propose to change the default. This starving already

happens in very simple/clean setups, unless you start to round-robin

all the channels. But again, this does not prevent out-of-order arrivals.


This is one thing where we have a completely different point of view. If
you need in-order arrival, you have to enforce it on the reading side with
something like Rock's stream aligner. The only case where the proposed
change will allow you some in-order guarantees is the local connection case
anyways, in all other cases you have unbounded latencies that do
reordering. As for stale data: if you expect buffers to be used on your
inputs, the receiving component must always read all data available (hence,
no "very old stale data"). Otherwise, you will always have some form of
buildup in the buffers, in a form you can't control. Finally, sharing a
single buffer allows one high-frequency connection to starve all others by
filling the buffer.


> I prefer the proposed solution since the default policy leads to a
> much more predictable data flow. For appropriately sizing buffers

for push channels in multi-rate component systems, we thought

about it to specify on the input port level the buffer size
> (there is already a default conn policy for each input port). A code

generator like orogen could easily calculate the largest buffer size

required between components and inform the input port, or stick to
> pull connections.
>

orogen only deals with single components. Syskit is the one inmpacted by
this change, and it is for me out of the picture to compute all possible
connections in all possible networks. Pull connections are not practical as
soon as you have a inter-host connection (i.e. remote connection) as they
break the reading part.

The bottom line is: the proposed change is really broken in the use case of
networks that are dynamic, which is my very use case. Making the new (or
the old) behaviour optional would be also hard because they really demand
different APIs (you can't keep a policy in the connection API when truly
the policy is per-port).


> I would definately not 'imply' different behaviours depending on
> buffer/data. After all, a data object is just a circular buffer of size 1.
>

I thought the same, but start to disagree. Data and buffer have (and should
have) different semantics. The fact that the data object is the same don't
change that.

Data connection: read() only returns the last sample received by the input
port
Buffer connection: read() returns all samples received by the input port
(apart from buffer size limitations).

Sylvain

--001a11c3288294673c05209c22d3
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-colo=
r:rgb(204,204,204);border-left-style:solid;padding-left:1ex">It doesn&#39;t=
 fix the out-of-order bug for the buffer-per-channel case.<br>
You can still starve=C2=A0a certain channel and get old data much later.=C2=
=A0</blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-le=
ft-style:solid;padding-left:1ex">That&#39;s why we propose=C2=A0to change t=
he=C2=A0default. This starving already=C2=A0</blockquote><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;bo=
rder-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">=
happens in very simple/clean setups,=C2=A0unless you start=C2=A0to round-ro=
bin=C2=A0</blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px=
 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);bor=
der-left-style:solid;padding-left:1ex">all the channels. But again, this do=
es not prevent=C2=A0out-of-order arrivals.</blockquote><div><br></div><div>=
This is one thing where we have a completely different point of view. If yo=
u need in-order arrival, you have to enforce it on the reading side with so=
mething like Rock&#39;s stream aligner. The only case where the proposed ch=
ange will allow you some in-order guarantees is the local connection case a=
nyways, in all other cases you have unbounded latencies that do reordering.=
 As for stale data: if you expect buffers to be used on your inputs, the re=
ceiving component must always read all data available (hence, no &quot;very=
 old stale data&quot;). Otherwise, you will always have some form of buildu=
p in the buffers, in a form you can&#39;t control. Finally, sharing a singl=
e buffer allows one high-frequency connection to starve all others by filli=
ng the buffer.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(2=
04,204,204);border-left-style:solid;padding-left:1ex">
I prefer the proposed solution since the default policy leads to a<br>
much more predictable=C2=A0data flow. For appropriately sizing buffers=C2=
=A0</blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-le=
ft-style:solid;padding-left:1ex">for push channels in=C2=A0multi-rate compo=
nent systems,=C2=A0we thought=C2=A0</blockquote><blockquote class=3D"gmail_=
quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-=
color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">about it t=
o specify on the input port level the buffer size<br>
(there is already a=C2=A0default conn policy for each input port). A code=
=C2=A0</blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p=
x 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border=
-left-style:solid;padding-left:1ex">generator like orogen=C2=A0could easily=
 calculate=C2=A0the largest buffer size=C2=A0</blockquote><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;bo=
rder-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">=
required between components and inform the=C2=A0input port, or stick to<br>
pull connections.<br></blockquote><div><br></div><div>orogen only deals wit=
h single components. Syskit is the one inmpacted by this change, and it is =
for me out of the picture to compute all possible connections in all possib=
le networks. Pull connections are not practical as soon as you have a inter=
-host connection (i.e. remote connection) as they break the reading part.=
=C2=A0</div><div><br></div><div>The bottom line is: the proposed change is =
really broken in the use case of networks that are dynamic, which is my ver=
y use case. Making the new (or the old) behaviour optional would be also ha=
rd because they really demand different APIs (you can&#39;t keep a policy i=
n the connection API when truly the policy is per-port).</div><div>=C2=A0</=
div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor=
der-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:sol=
id;padding-left:1ex">
I would definately not &#39;imply&#39; different behaviours depending on<br=
>
buffer/data. After=C2=A0all, a data object is just a circular buffer of siz=
e 1.<br></blockquote><div><br></div><div>I thought the same, but start to d=
isagree. Data and buffer have (and should have) different semantics. The fa=
ct that the data object is the same don&#39;t change that.</div><div><br></=
div><div>Data connection: read() only returns the last sample received by t=
he input port<br></div><div>Buffer connection: read() returns all samples r=
eceived by the input port (apart from buffer size limitations).</div><div><=
br></div><div>Sylvain</div></div>
</div>

--001a11c3288294673c05209c22d3--

--===============0193899577==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

-- 
Orocos-Dev mailing list
[email protected]
http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev

--===============0193899577==--