Re: Limitations with v2 port implementation

On Apr 08, 2015, at 10:23, S Roderick <[email protected]> wrote:

> On Apr 08, 2015, at 10:17, Sylvain Joyeux <[email protected]> wrote:
> =

>>> We=92ve not looked at the effect of different lock policies, but I=92m =
very reluctant to start making those kind of changes unless really necessar=
y. And in all honesty that feels like a solution to a solution. The perform=
ance issue is fundamentally driven by the 1-to-N fanout from output ports t=
o input ports. Certainly the locking solution may/will improve performance,=
 but the fundamental issue remains.
>> =

>> I asked because it would point to where the problem might be, *not* as
>> a solution to your particular problem (i.e. test the hypothesis of
>> "the lock free implementation is the major problem).
> =

> Understood
> =

>> Just another shot in the dark: do you build with optimizations on ?
> =

> Yes. :-)
> =

>> Apart from things like configuration, can you detail some other
>> user-cases for the original v1 behaviour ? I made the original push
>> for the v2 channels *because* the v1 semantic were more than awkward
>> (and harmful) when building our systems, so I am very curious to see
>> what I missed=85
> =

> I=92ll put something together later for you Sylvain (gotta go for now). I=
t=92s a good discussion to have. And I don=92t know that anything was reall=
y =93missed=94, as much as there are other equally valid ways to do things.

Our system was built from the ground up using v1, including working around =
some of its awkward parts that the v2 ports with flow status do nicely reme=
dy. For this discussion, I=92d divide our system into four parts

1) a small number of processing pipelines (think images) for which flow sta=
tus works well to "wake up=94 components in sequence when new data arrives.=
 In v1 we made small coordinators to do this instead - fugly.
2) some asynchronous higher-level processing components
3) some other misc. components like logging, reporting, etc - irrelevant
4) but the majority of our system is a synchronous set of components that a=
re woken up by a hardware signal; the appropriate computation components ar=
e sequenced by a high level coordinating supervisor; and, then new commands=
 are sent to hardware.

Flow status ports are great for 1), largely irrelevant for 3), some times p=
ainful for 2), and nothing but trouble for 4) as a rule. The problem with 4=
) is that the =93flow=94 is coordinated at the component level, not at the =
data level (I suspect that is one of the big differences between our system=
s). The motion control pipelines that are part of 4) are sequenced/coordina=
ted by component - there was no practical concept of waking up for data arr=
ival in v1 (ignoring EventPorts for technical reasons), so we did it this w=
ay instead. And within 4) there are a lot of fan-out and fan-in situations =
with multiple input-to-output ports, and in some cases, multiple-input-to-m=
ultiple-outputs - in this case individual connections are trouble.

In terms of use cases
- static configuration data, usually one output to many input
- communication between the async. components in 2) and the synch. componen=
ts in 4) don=92t care about flow (at a system level - remember the system s=
emantics were designed for v1) - they just want the last value written
- multiple outputs to one input, where the outputs can be switched between =
on the fly. And some of the outputs are synch. while some are asynch. Here,=
 flow status is just nasty, and we=92ve had to add a bunch of explicit clea=
r calls, and the like, into our system to effectively flush (or similar) al=
l data when certain mode switches occur. Because the upstream components ar=
e coordinated, we only want the last value written (which is, by definition=
, from the last component executed)
- performance - some of our systems run really close to deadline, and the e=
xtra few % of updating multiple connection objects is too much.

In a more theoretical sense, our components were (mostly) written to push t=
he responsibility of when and whom wrote input data up to a higher level (i=
.e. a coordinator). They all depend on a =93I just want the last/latest val=
ue available on my input port=94 mentality. I still maintain that this is v=
alid, even with v2 flow ports, as it decouples the inside of the component =
from any other components, and puts the responsibility for coordination on =
the system designer. In reality, we=92ve now in v2 got a lot more coupling =
of the inside of our components with other peer components, because v2 port=
s (in *some* ways) require that the end user be more aware of when and whom=
 wrote data. Again, v2 ports are just different - I=92m not saying they=92r=
e good, bad, or broken.

What I don=92t want to get into here is a lot of =93well just redesign your=
 system to =85=94 kind of discussion. We have a large system with a lot of =
legacy in its architecture and design. No doubt, if we were to re-design fr=
om scratch with only v2 in mind we=92d come up with a different result. Wha=
t I=92m hoping for here is discussion on the general concept of whether the=
 v2 port implementation (flow status with a connection object per input/out=
put combination) solves the general use cases that we inherited from v1, an=
d which I still feel are equally as valid as using flow status type ports.

Hope this helps, Sylvain (and everyone else).

HTH
S


-- =

Orocos-Dev mailing list
[email protected]
http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev