Re: Updated dataflow semantics for RTT
S Roderick <[email protected]> Mon, 28 Sep 2015 20:14:39 +0000 (GMT)
| Newsgroups | gmane.science.robotics.orocos.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============1656807144== Content-type: multipart/alternative; boundary="Boundary_(ID_RnyRWOg5Yibo/kl68jvI2w)" --Boundary_(ID_RnyRWOg5Yibo/kl68jvI2w) Content-type: text/plain; charset=utf-8; format=flowed Content-transfer-encoding: quoted-printable On Sep 28, 2015, at 03:44 PM, Sylvain Joyeux <[email protected]> wrot= e:=0A=0ASome classic producer/consumer examples=0A- a camera captures imag= es and sends them on to a processing pipeline (that=0Ais all data object c= onnections). A separate connection stores the camera=0Aimages in a buffer,= which are dumped to disk.=0A- data is read (non-real-time) from a file an= d pushed into a buffer for=0Aprocessing (real-time)=0A- data samples comin= g from a remote source (non-real-time) is buffered on=0Aarrival into the d= eployer for later processing (real-time).=0A=0AAsked the wrong question ..= . thanks for answering it perfectly ;-)=0A=0AAnyways, what I wanted is a c= ase where:=0A- an input port is connected to a bunch of output ports *usin= g=0Abuffers* (I got the problem with data policies)=0A- the overall behavi= or is governed by maintaining the overall=0Aincoming sample order (i.e. th= at if a sample of output port O_1=0Aarrives before a sample of output port= O_2, that order needs to be=0Areflected on read())=0A=C2=A0=0AI think tha= t all of the buffer uses in our system are classic single producer/single = consumer connections. =C2=A0I can't think of any N:M set of connections us= ing buffers.=0A=0AI may be missing it, but the use-cases above don't cover= that, do they ?=0A=0A> Agreed, but this choice is currently bundled with = having a buffer per=0A> channel or one per input port.=0A=0AI don't follow= , Sylvain. Bundled in the existing implementation or the=0Aproposed change= s?=0A=0AIn the proposed change (current v2 does not have the option of hav= ing=0Aa single buffer per input port). Until now, it is clear to me that=0A= this bundling is both Johannes and Peter's intent (see Peter's email=0Aabo= ve). It does not seem to be yours, though.=0A=C2=A0=0AWell, yes and no. I'= m not going to throw=C2=A0Johannes and=C2=A0Peter under the bus. A lot of = this work is at our urging, and we've been involved with formulating and e= valuating possible solutions. They have put together a=C2=A0solution=C2=A0= that=C2=A0they believe solves some of the issues that we, and others, have= bought up. They have also come to this proposed solution by actually impl= ementing (some of) it and running into quite a few corner cases, etc.=0A=0A= > If you put the buffer per-port, you end up with one policy per port.=0A>= Port-driven triggering (event ports) are governed by the data object=0A> = (i.e. per-port), data object size and type becomes per-port as well.=0A> W= hat's left in the connection policy is the info necessary to reach=0A> the= other side (i.e. the topic name in ROS), which means that it is=0A> not a= policy anymore but a "remote endpoint"=0A=0AFrom your comment above "Port= -driven triggering (event ports) =E2=80=A6"=0A=0ABadly ordered bits of sen= tences. What I meant is:=0A=0AIf you put the buffer per-port, you end up w= ith one policy per port. Data object size and type becomes per-port. If us= ing port-driven triggering (event ports), the triggering is governed by th= e data object and is therefore also becoming per-port.=0A=0ALooking at Sec= tion 5.1 of Johannes' google doc, something looks wrong or I'm=0Arememberi= ng previous discussions incorrectly. I was expecting push/pull vs=0Ashared= /private to be truly orthogonal, but the push/private figure says=0Adiffer= ently. Johannes? Peter?=0A=0AThey already clarified that for me. Basically= , "private" does not=0Anecessarily means "per channel". On a push connecti= on, "private" for=0Athem means "per-port".=0A=C2=A0=0AIn my head, private = means per "connection" (I think that is your "channel" also). The proposed= solution may not have that for good reasons, which I'll let them state.=0A= =0AAgain, my intent was to add more v1-like semantics to the system, as we= ll as=0Ahave more control on where latency occurred. Not to replace any as= pect of=0Athe v2 semantics.=0A=0AI got that from your previous email alrea= dy, but it seems that it's=0Anot Johannes/Peter's intent=0A=C2=A0=0AAgain,= they're trying to fix multiple issues in one go (and all the issues are p= retty inter-related). They're also dealing with some pretty horrific corne= r cases, and they're doing the best they can.=0A=0AHTH=0AS=0A=0A= --Boundary_(ID_RnyRWOg5Yibo/kl68jvI2w) Content-type: multipart/related; boundary="Boundary_(ID_u4a+DCGYhWduB4UHuwg1NQ)"; type="text/html" --Boundary_(ID_u4a+DCGYhWduB4UHuwg1NQ) Content-type: text/html; charset=utf-8 Content-transfer-encoding: quoted-printable <html><body><div><span style=3D"line-height: 1.5;">On Sep 28, 2015, at 03:44 PM, Sylva= in Joyeux <[email protected]> wrote:</span></div><div><br></div= ><div><blockquote type=3D"cite"><div class=3D"msg-quote"><div class=3D"_st= retch"><blockquote class=3D"quoted-plain-text" type=3D"cite">Some classic = producer/consumer examples</blockquote><blockquote class=3D"quoted-plain-t= ext" type=3D"cite">- a camera captures images and sends them on to a proce= ssing pipeline (that</blockquote><blockquote class=3D"quoted-plain-text" t= ype=3D"cite">is all data object connections). A separate connection stores= the camera</blockquote><blockquote class=3D"quoted-plain-text" type=3D"ci= te">images in a buffer, which are dumped to disk.</blockquote><blockquote = class=3D"quoted-plain-text" type=3D"cite">- data is read (non-real-time) f= rom a file and pushed into a buffer for</blockquote><blockquote class=3D"q= uoted-plain-text" type=3D"cite">processing (real-time)</blockquote><blockq= uote class=3D"quoted-plain-text" type=3D"cite">- data samples coming from = a remote source (non-real-time) is buffered on</blockquote><blockquote cla= ss=3D"quoted-plain-text" type=3D"cite">arrival into the deployer for later= processing (real-time).</blockquote><span class=3D"body-text-content"><sp= an class=3D"body-text-content"><br>Asked the wrong question ... thanks for= answering it perfectly ;-)<br><br>Anyways, what I wanted is a case where:= <br> - an input port is connected to a bunch of output ports *using<br>buf= fers* (I got the problem with data policies)<br> - the overall behavior is= governed by maintaining the overall<br>incoming sample order (i.e. that i= f a sample of output port O_1<br>arrives before a sample of output port O_= 2, that order needs to be<br>reflected on read())</span></span></div></div= ></blockquote><span> </span></div><div>I think that all of the buffer= uses in our system are classic single producer/single consumer connection= s. I can't think of any N:M set of connections using buffers.</div><= div><br></div><div><blockquote type=3D"cite"><div class=3D"msg-quote"><div= class=3D"_stretch"><span class=3D"body-text-content"><span class=3D"body-= text-content">I may be missing it, but the use-cases above don't cover tha= t, do they ?<br><br></span></span><blockquote class=3D"quoted-plain-text" = type=3D"cite">> Agreed, but this choice is currently bundled with havin= g a buffer per</blockquote><blockquote class=3D"quoted-plain-text" type=3D= "cite">> channel or one per input port.</blockquote><blockquote class=3D= "quoted-plain-text" type=3D"cite"><br></blockquote><blockquote class=3D"qu= oted-plain-text" type=3D"cite">I don't follow, Sylvain. Bundled in the exi= sting implementation or the</blockquote><blockquote class=3D"quoted-plain-= text" type=3D"cite">proposed changes?</blockquote><span class=3D"body-text= -content"><span class=3D"body-text-content"><br>In the proposed change (cu= rrent v2 does not have the option of having<br>a single buffer per input p= ort). Until now, it is clear to me that<br>this bundling is both Johannes = and Peter's intent (see Peter's email<br>above). It does not seem to be yo= urs, though.</span></span></div></div></blockquote><span> </span></di= v><div>Well, yes and no. I'm not going to throw <span style=3D"line-h= eight: 1.5;">Johannes and Peter under the bus. A lot of this work is = at our urging, and we've been involved with formulating and evaluating pos= sible solutions. They have put together a </span>solution<span style=3D= "line-height: 1.5;"> </span>that<span style=3D"line-height: 1.5;">&nb= sp;they believe solves some of the issues that we, and others, have bought= up. They have also come to this proposed solution by actually implementin= g (some of) it and running into quite a few corner cases, etc.</span></div= ><div><br></div><div><blockquote type=3D"cite"><div class=3D"msg-quote"><d= iv class=3D"_stretch"><blockquote class=3D"quoted-plain-text" type=3D"cite= ">> If you put the buffer per-port, you end up with one policy per port= .</blockquote><blockquote class=3D"quoted-plain-text" type=3D"cite">> P= ort-driven triggering (event ports) are governed by the data object</block= quote><blockquote class=3D"quoted-plain-text" type=3D"cite">> (i.e. per= -port), data object size and type becomes per-port as well.</blockquote><b= lockquote class=3D"quoted-plain-text" type=3D"cite">> What's left in th= e connection policy is the info necessary to reach</blockquote><blockquote= class=3D"quoted-plain-text" type=3D"cite">> the other side (i.e. the t= opic name in ROS), which means that it is</blockquote><blockquote class=3D= "quoted-plain-text" type=3D"cite">> not a policy anymore but a "remote = endpoint"</blockquote><span class=3D"body-text-content"><span class=3D"bod= y-text-content"><br></span></span><blockquote class=3D"quoted-plain-text" = type=3D"cite">From your comment above "Port-driven triggering (event ports= ) =E2=80=A6"</blockquote><span class=3D"body-text-content"><span class=3D"= body-text-content"><br>Badly ordered bits of sentences. What I meant is:<b= r><br></span></span><blockquote class=3D"quoted-plain-text" type=3D"cite">= If you put the buffer per-port, you end up with one policy per port. Data = object size and type becomes per-port. If using port-driven triggering (ev= ent ports), the triggering is governed by the data object and is therefore= also becoming per-port.</blockquote></div></div></blockquote></div><div><= blockquote type=3D"cite"><div class=3D"msg-quote"><div class=3D"_stretch">= <span class=3D"body-text-content"><span class=3D"body-text-content"><br></= span></span><blockquote class=3D"quoted-plain-text" type=3D"cite">Looking = at Section 5.1 of Johannes' google doc, something looks wrong or I'm</bloc= kquote><blockquote class=3D"quoted-plain-text" type=3D"cite">remembering p= revious discussions incorrectly. I was expecting push/pull vs</blockquote>= <blockquote class=3D"quoted-plain-text" type=3D"cite">shared/private to be= truly orthogonal, but the push/private figure says</blockquote><blockquot= e class=3D"quoted-plain-text" type=3D"cite">differently. Johannes? Peter?<= /blockquote><span class=3D"body-text-content"><span class=3D"body-text-con= tent"><br>They already clarified that for me. Basically, "private" does no= t<br>necessarily means "per channel". On a push connection, "private" for<= br>them means "per-port".</span></span></div></div></blockquote><span>&nbs= p;</span></div><div>In my head, private means per "connection" (I think th= at is your "channel" also). The proposed solution may not have that for go= od reasons, which I'll let them state.</div><div><br><blockquote type=3D"c= ite"><div class=3D"msg-quote"><div class=3D"_stretch"><blockquote class=3D= "quoted-plain-text" type=3D"cite">Again, my intent was to add more v1-like= semantics to the system, as well as</blockquote><blockquote class=3D"quot= ed-plain-text" type=3D"cite">have more control on where latency occurred. = Not to replace any aspect of</blockquote><blockquote class=3D"quoted-plain= -text" type=3D"cite">the v2 semantics.</blockquote><span class=3D"body-tex= t-content"><span class=3D"body-text-content"><br>I got that from your prev= ious email already, but it seems that it's<br>not Johannes/Peter's intent<= /span></span></div></div></blockquote><span> </span></div><div>Again,= they're trying to fix multiple issues in one go (and all the issues are p= retty inter-related). They're also dealing with some pretty horrific corne= r cases, and they're doing the best they can.</div><div><br></div><div>HTH= <br>S<br><br></div></body></html>= --Boundary_(ID_u4a+DCGYhWduB4UHuwg1NQ)-- --Boundary_(ID_RnyRWOg5Yibo/kl68jvI2w)-- --===============1656807144== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline -- Orocos-Dev mailing list [email protected] http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev --===============1656807144==--