Re: Updated dataflow semantics for RTT
Johannes Meyer <[email protected]> Mon, 28 Sep 2015 21:52:05 +0200
| Newsgroups | gmane.science.robotics.orocos.devel |
|---|---|
| Message-ID | <CABowFJFe4HxJNEw0Aq_iM1+KUDnmAopSTic6SMnBfO1C5L-+0g@mail.gmail.com> |
--===============1511504056==
Content-Type: multipart/alternative; boundary=001a11349cee5e621e0520d409d9
--001a11349cee5e621e0520d409d9
Content-Type: text/plain; charset=UTF-8
Hi,
thanks for all your input and ideas and I agree with Herman that the
"one-framework-for-everything" approach of RTT is probably not how you
would start a new project nowadays. But I would like to support Stephen's
call to focus this discussion on the technical aspects of how to improve
the existing implementation in as small as possible ("incremental") steps
that add support for some new use cases that are not covered yet (but have
been partially covered by v1).
Maintaining backwards compatibility is definitely the way to go and we can
conclude that the new default semantics (especially per-connection vs.
per-input buffers) have to be the same as in previous v2 releases. I hope
you find it acceptable that a new version would certainly be not
ABI-compatible and breaks some internal API calls that are typically not
used directly by a user. This might break more advanced things like custom
transport implementations. I fear that these kind of changes cannot be
avoided completely, especially for the ConnFactory and ChannelElement
classes.
We should not limit the use cases of RTT on the core API level, like
enforcing per-input storage objects for data connections and per-connection
storage for buffer connections, but only add new use cases. It is more a
question of having good documentation and choosing reasonable defaults that
cover as many applications as possible, but at the end the application
designer has to have the freedom to deploy the application with whatever
connection semantics it demands for.
In the following I try to wrap up and comment on some of the concerns that
came up during the discussion so far, and suggest compromise solutions
which have been at least partially discussed with Peter:
*1) There is definitely a need for per-input port buffers AND for
per-connection buffers - INDEPENDENT of whether the data/buffer object is
installed on the reader and writer side.*
Indeed, according to the original proposal the questions of at what side of
a remote connection the data object will be installed and whether there
will be a per-input or a per-connection buffer cannot specified separately.
Peter and I could not imagine why you would want separate buffers if the
data is already available locally anyways (in the push case) and we still
think that at least for data connections the current per-connection
implementation can be considered as broken. Even with readNewest() there is
no guarantee that you get the latest sample written. On the other hand,
Sylvain correctly stated that there are no guarantees anyway as soon as
remote connections (or any non-realtime transport) are considered and the
only true solution is proper timestamping and synchronization. But I think
we agree that for most deployments with only local connections the
assumption that the latest sample is the last written sample holds and that
timestamping is beyond the scope of RTT and can be left to user code or
more advanced tools like the Rock stream aligner.
Even with the proposed changes (let's call them v3 semantics as a working
title) the connection policy is still primarily describing the connection,
even if *some* settings limit the set of allowed policies for other
connections, making *some* other settings de-facto a per-port setting. This
would be primarily true for input ports and not at all for output ports,
unless it has a connection of the new shared type. But it is not an option
to associate the connection policy *only* with a port because not *all*
possible connections imply per-port buffers. Specifying a connection policy
on a writer/reader pair base as it is now and let certain invalid
connection attempts fail is still the most general approach.
*Proposal:* In order to not break existing applications, we introduce a new
ConnPolicy field "read_policy" that toggles between per-input and
per-connection buffers, instead of implying that by the existing "pull"
flag for remote connections. The read_policy would be an enum with two
possible values for now, "SingleInputBuffer" and "PreferCurrentConnection".
The reason why we propose an enum here is that you could also think of
other policies on how to select the next channel to read from in the
per-connection case, e.g. RoundRobin, PreferLastSignaled, etc. The current
v2 default would correspond to PreferCurrentConnection, as the channel that
was last read with new data is preferred by the input port and only then it
polls all other connections in the order they have been made. We have not
thought this idea to the end yet and the names of the constants are still
under discussion. We do not want to call the input buffers "shared",
because the read policy is independent of the new concept of a "shared
connection" where multiple input ports read from the same buffer. Or
someone can come up with a better name for this type of connection...
*2) Should the new per-input port buffers become the new default or not
(SingleInputBuffer read policy)?*
With that respect the conclusion is that backwards compatibility is more
important than the new "features" and the default parameters for the
current ConnPolicy class should not be touched. However, there are
different ways to tackle this:
2.1 Make it a compile-time option.
2.2 Only add *new* fields to the ConnPolicy class without changing the
default parameter values in the constructor and static methods, and a new
class ConnPolicy2 or ConnPolicy3 that inherits from ConnPolicy and might
have other default parameters or another C++ API (I am thinking of a
named-parameter <https://isocpp.org/wiki/faq/ctors#named-parameter-idiom>
based API) and can also be used for scripting. Internally, the connection
factory would work with const references to the base class only.
2.3 We add a DefaultConnPolicy attribute to the GlobalsRepository. This
application-wide (per process) policy instance will be copied to newly
constructed ConnPolicy instances and the constructors and static methods
only overwrite the respective parameters.
It might be obsoleted by the above proposal, but in the meantime I added a
compile-time option RTT_V2_COMPATIBILITY_MODE to my working branch (see
ef99f2b
<https://github.com/orocos-toolchain/rtt/commit/ef99f2bfa2c01f17431f5a319f117018081ffd91>),
that,
if set, reverts to the current v2 per-connection model, but lets RTT print
meaningful warning messages in the case a new connection would be invalid
with v3 semantics because of conflicts with other connections to the same
port. Such a flag could smooth the transition phase and help application
designers to find the minimal set of connections that require connection
policy changes.
There is one small exception from the backwards-compatibility rule, that I
think is worth discussing: The static method
static ConnPolicy data(int lock_policy = LOCK_FREE, bool init_connection =
true, bool pull = false);
initializes the init_connection field to true by default, which was kind of
surprising me, because it means that every new connection to an output port
with this policy will overwrite a sample previously written by another
port. Is this what we really want as a default? It is also not consistent
with the constructor called as
ConnPolicy(DATA)
which would initialize the init_connection flag to false. The static method
data(...) is only used in the
OutputPortInterface::createDataConnection(...) method from within RTT and
OCL itself.
By the way, the init_connection flag is a good example where the API offers
some degree of freedom to the application builder that semantically does
not make sense at all for certain combinations, like in combination with a
buffer connection. It is still not forbidden to set the flag, but it might
be worth a warning log message if a connection is created like this.
*3) How to add the "shared" concept within the current API, or to use a
"new" API?*
This point was brought up by Janosch, who asked for "a more explicit
interface for shared connections", with the main argument that it is
fundamentally different from a normal connection and a "connect port A to B
with policy C" kind of API does not fit well to that concept. That's true.
We considered to use the createStream(...) API first, with a name_id string
that identifies the shared buffer to connect to. This would actually fit
better to the shared connection model, but it still enforces the same
buffer type policies for all connections. We withdraw that idea because the
existing API is more general and disallowing certain policy combinations is
required anyway for private push connections with a single input buffer. At
least in this case it cannot break existing applications. I do not think
that there is a need for a completely new API only for shared connections.
My current implementation in the updated-dataflow-semantics branch
<https://github.com/orocos-toolchain/rtt/compare/master...updated-dataflow-semantics>
is
as follows, and new/other ideas are always welcome:
- Shared connection objects (the shared data object or buffered) are added
to a process-wide registry. Every shared connection is identified by a
unique string, either set explictly in the name_id field of the ConnPolicy
or assigned automatically, in which case the name of the connection is
returned in exactly this field (name_id is and always was declared as
mutable).
- For new connections with the name_id set, the shared connection is looked
up in the registry, or created as new. As a consequence, if the same
ConnPolicy instance is used to make a second connection, even if name_id
was not set explicitly before, you will always connect to the same
connection object.
- For new connections with no name_id set, the port objects provide an API
to return a pointer to an existing shared connection if they are already
connected to an existing one, and only the non-connected port will be
added. So ports can be connected in any order to each other, as long as
there is always one of the two ports already connected. Otherwise the
name_id has to be set or a new connection instance would be created.
Perhaps this one could be dropped because it it too implicit and does not
work very well for remote connections...
- If an existing connection was found, the buffer policies (type, size and
locking policy) have to match.
- All ports will only store a single pointer to the shared connection
instance in their connection list (in internal::ConnectionManager) and
cannot know directly which other ports are connected.
- For remote input ports with a shared connection, only a proxy is created
locally and can be looked up in the shared connection repository. Other
ports will only connect to either the proxy or the real shared connection
instance, but never build a new remote channel once the proxy has been
created for the first remote connection (works in both directions).
- All connect and disconnect calls to shared connections are logged with
log level Debug to the RTT logger, so at least there is a manual way to
check whether the resulting connections have been made as expected.
Graphical tools like the rtt_dot_service
<https://github.com/orocos-toolchain/rtt_dot_service> could be patched
later to visualize shared connections correctly.
*4) Whether to encode the result of a write() in existing enums or not?*
Janosch asked for introducing a separate enum for the return value of
write(...) calls, in order to not break or trigger compiler warnings for
existing code that already uses the FlowStatus enum in a switch statement.
Another minor issue with a single enum is the default value of FlowStatus
attributes, which can be only meaningful for one use case.
That's also fine for me and I will update the document and implementation
accordingly (add a new enum WriteStatus). Note that the evaluation in
boolean context is counter-intuitive in this case, other than for
FlowStatus, because the value WriteSuccess (=0) value evaluates to false
and all error cases (WriteFailure and NotConnected) evaluate to true.
I hope I did not forget anything. This mail already became much longer than
expected. I am sorry.
I also did some performance measurements of read and write calls last week
in terms of absolute time, CPU time, number of assignments and so on of the
current, the new and even the v1 data flow implementations in different
scenarios. I plan to come up with some results during this week. The tests
revealed some bugs and performance caveats, and some of them are already
present since the early days of RTT v2, e.g. that data samples might be
lost (never read as NewData) because the read flag is not lock-protected or
stored within the lock-free data structure. I am preparing a pull request
to the current master branch, but it will also break the API of class
base::DataObjectInterface.
Best regards,
Johannes
On Mon, Sep 28, 2015 at 3:36 PM, S Roderick <[email protected]> wrote:
>
> Johannes' proposed changes are focussed on
> a) push vs pull, or specifying whether the data/buffer object is on the
> reader or writer side. This is very useful for system designers dealing
> with significant time delay.
> b) private vs shared, whether a connection is per input/output port pair
> or whether the connection is shared between multiple input/output ports.
> c) mandatory, primarily intended to support knowing whether writes to a
> buffer succeeded (so that traditional producer/consumer relationships have
> some knowledge about the buffer state)
>
> It appears to me that most people are worried about changing the defaults,
> more than they're worried about the actual proposed changes. Actual
> technical concerns with the proposed changes appear to be
>
> 1) whether to encode the result of a write() in existing enums or not
> 2) how to add the "shared" concept within the current API, or to use a
> "new" API
> 3) changing the RTT defaults, which then affect an application's semantics.
>
> Did I miss any technical issues?
>
>
> My huge personal caveat to all of this is that I do *not* think we should
> change the current system semantics in introducing this change. Too many
> Orocos changes have broken backwards compatibility and forced system
> designers and implementors to change their system in response. I don't
> think we should force that on users without an incredibly good reason.
> Otherwise it breeds frustration in the community.
>
> If there is some way to introduce the proposed changes to the system, but
> to not change the behavior of current systems, then I think that is a win
> for all. There does seem to be some general agreement that there are valid
> use cases that the current port implementation handles poorly, and that
> this proposed approach does fix some of those. The proposed approach also
> improves the situation for systems that don't want or need to use the flow
> status approach. Both are equally valid approaches - IMHO it would be
> beneficial to the community as a whole if RTT supported both approaches.
>
> Cheers
> Stephen
>
>
>
> --
> Orocos-Dev mailing list
> [email protected]
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>
--001a11349cee5e621e0520d409d9
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Hi,<div><br></div><div>thanks for all your input and ideas=
and I agree with Herman that the "one-framework-for-everything" =
approach of RTT is probably not how you would start a new project nowadays.=
But I would like to support Stephen's call to focus this discussion on=
the technical aspects of how to improve the existing implementation in as =
small as possible (<span style=3D"font-size:12.8px">"incremental"=
)=C2=A0</span>steps that add support for some new use cases that are not co=
vered yet (but have been partially covered by v1).</div><div><br></div><div=
>Maintaining backwards compatibility is definitely the way to go and we can=
conclude that the new default semantics (especially per-connection vs. per=
-input buffers) have to be the same as in previous v2 releases. I hope you =
find it acceptable that a new version would certainly be not ABI-compatible=
and breaks some internal API calls that are typically not used directly by=
a user. This might break more advanced things like custom transport implem=
entations. I fear that these kind of changes cannot be avoided completely, =
especially for the ConnFactory and ChannelElement classes.</div><div><br></=
div><div>We should not limit the use cases of RTT on the core API level, li=
ke enforcing per-input storage objects for data connections and per-connect=
ion storage for buffer connections, but only add new use cases. It is more =
a question of having good documentation and choosing reasonable defaults th=
at cover as many applications as possible, but at the end the application d=
esigner has to have the freedom to deploy the application with whatever con=
nection semantics it demands for.</div><div><br></div><div>In the following=
I try to wrap up and comment on some of the concerns that came up during t=
he discussion so far, and suggest compromise solutions which have been at l=
east partially discussed with Peter:</div><div><br></div><div><b>1) There i=
s definitely a need for per-input port buffers AND for per-connection buffe=
rs - INDEPENDENT of whether the data/buffer object is installed on the read=
er and writer side.</b></div><div><br></div><div>Indeed, according to the o=
riginal proposal the questions of at what side of a remote connection the d=
ata object will be installed and whether there will be a per-input or a per=
-connection buffer cannot specified separately. Peter and I could not imagi=
ne why you would want separate buffers if the data is already available loc=
ally anyways (in the push case) and we still think that at least for data c=
onnections the current per-connection implementation can be considered as b=
roken. Even with readNewest() there is no guarantee that you get the latest=
sample written. On the other hand, Sylvain correctly stated that there are=
no guarantees anyway as soon as remote connections (or any non-realtime tr=
ansport) are considered and the only true solution is proper timestamping a=
nd synchronization. But I think we agree that for most deployments with onl=
y local connections the assumption that the latest sample is the last writt=
en sample holds and that timestamping is beyond the scope of RTT and can be=
left to user code or more advanced tools like the Rock stream aligner.</di=
v><div><br></div><div>Even with the proposed changes (let's call them v=
3 semantics as a working title) the connection policy is still primarily de=
scribing the connection, even if *some* settings limit the set of allowed p=
olicies for other connections, making *some* other settings de-facto a per-=
port setting. This would be primarily true for input ports and not at all f=
or output ports, unless it has a connection of the new shared type. But it =
is not an option to associate the connection policy *only* with a port beca=
use not *all* possible connections imply per-port buffers. Specifying a con=
nection policy on a writer/reader pair base as it is now and let certain in=
valid connection attempts fail is still the most general approach.</div><di=
v><br></div><div><b>Proposal:</b> In order to not break existing applicatio=
ns, we introduce a new ConnPolicy field "read_policy" that toggle=
s between per-input and per-connection buffers, instead of implying that by=
the existing "pull" flag for remote connections. The read_policy=
would be an enum with two possible values for now, "SingleInputBuffer=
" and "PreferCurrentConnection". The reason why we propose a=
n enum here is that you could also think of other policies on how to select=
the next channel to read from in the per-connection case, e.g. RoundRobin,=
PreferLastSignaled, etc. The current v2 default would correspond to Prefer=
CurrentConnection, as the channel that was last read with new data is prefe=
rred by the input port and only then it polls all other connections in the =
order they have been made. We=C2=A0have not thought this idea to the end ye=
t and the names of the constants are still under discussion. We do not want=
to call the input buffers "shared", because the read policy is i=
ndependent of the new concept of a "shared connection" where mult=
iple input ports read from the same buffer. Or someone can come up with a b=
etter name for this type of connection...</div><div><br></div><div><b><br><=
/b></div><div><b>2) Should the new per-input port buffers become the new de=
fault or not (SingleInputBuffer read policy)?</b><br></div><div><b><br></b>=
</div><div>With that respect the conclusion is that backwards compatibility=
is more important than the new "features" and the default parame=
ters for the current ConnPolicy class should not be touched. However, there=
are different ways to tackle this:</div><div><br></div><div>2.1 Make it a =
compile-time option.</div><div><br></div><div>2.2 Only add *new* fields to =
the ConnPolicy class without changing the default parameter values in the c=
onstructor and static methods, and a new class ConnPolicy2 or ConnPolicy3 t=
hat inherits from ConnPolicy and might have other default parameters or ano=
ther C++ API (I am thinking of a <a href=3D"https://isocpp.org/wiki/faq/cto=
rs#named-parameter-idiom">named-parameter</a> based API) and can also be us=
ed for scripting. Internally, the connection factory would work with const =
references to the base class only.</div><div><br></div><div>2.3 We add a De=
faultConnPolicy attribute to the GlobalsRepository. This application-wide (=
per process) policy instance will be copied to newly constructed ConnPolicy=
instances and the constructors and static methods only overwrite the respe=
ctive parameters.</div><div><br></div><div><div>It might be obsoleted by th=
e above proposal, but in the meantime I added a compile-time option RTT_V2_=
COMPATIBILITY_MODE to my working branch (see=C2=A0<a href=3D"https://github=
.com/orocos-toolchain/rtt/commit/ef99f2bfa2c01f17431f5a319f117018081ffd91">=
ef99f2b</a>),=C2=A0that, if set, reverts to the current v2 per-connection m=
odel, but lets RTT print meaningful warning messages in the case a new conn=
ection would be invalid with v3 semantics because of conflicts with other c=
onnections to the same port. Such a flag could smooth the transition phase =
and help application designers to find the minimal set of connections that =
require connection policy changes.</div><div><br></div></div><div>There is =
one small exception from the backwards-compatibility rule, that I think is =
worth discussing: =C2=A0The static method</div><div><br></div><div><span st=
yle=3D"background-color:rgb(255,255,255)"><span class=3D"" style=3D"color:r=
gb(167,29,93);font-family:Consolas,'Liberation Mono',Menlo,Courier,=
monospace;font-size:12px;line-height:16.8px;white-space:pre">static</span><=
span style=3D"color:rgb(51,51,51);font-family:Consolas,'Liberation Mono=
',Menlo,Courier,monospace;font-size:12px;line-height:16.8px;white-space=
:pre"> ConnPolicy </span><span class=3D"" style=3D"color:rgb(121,93,163);fo=
nt-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-s=
ize:12px;line-height:16.8px;white-space:pre">data</span><span style=3D"colo=
r:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Menlo,Courie=
r,monospace;font-size:12px;line-height:16.8px;white-space:pre">(</span><spa=
n class=3D"" style=3D"color:rgb(167,29,93);font-family:Consolas,'Libera=
tion Mono',Menlo,Courier,monospace;font-size:12px;line-height:16.8px;wh=
ite-space:pre">int</span><span style=3D"color:rgb(51,51,51);font-family:Con=
solas,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;line=
-height:16.8px;white-space:pre"> lock_policy =3D LOCK_FREE, </span><span cl=
ass=3D"" style=3D"color:rgb(167,29,93);font-family:Consolas,'Liberation=
Mono',Menlo,Courier,monospace;font-size:12px;line-height:16.8px;white-=
space:pre">bool</span><span style=3D"color:rgb(51,51,51);font-family:Consol=
as,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;line-he=
ight:16.8px;white-space:pre"> init_connection =3D </span><span class=3D"" s=
tyle=3D"color:rgb(0,134,179);font-family:Consolas,'Liberation Mono'=
,Menlo,Courier,monospace;font-size:12px;line-height:16.8px;white-space:pre"=
>true</span><span style=3D"color:rgb(51,51,51);font-family:Consolas,'Li=
beration Mono',Menlo,Courier,monospace;font-size:12px;line-height:16.8p=
x;white-space:pre">, </span><span class=3D"" style=3D"color:rgb(167,29,93);=
font-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font=
-size:12px;line-height:16.8px;white-space:pre">bool</span><span style=3D"co=
lor:rgb(51,51,51);font-family:Consolas,'Liberation Mono',Menlo,Cour=
ier,monospace;font-size:12px;line-height:16.8px;white-space:pre"> pull =3D =
</span><span class=3D"" style=3D"color:rgb(0,134,179);font-family:Consolas,=
'Liberation Mono',Menlo,Courier,monospace;font-size:12px;line-heigh=
t:16.8px;white-space:pre">false</span><span style=3D"color:rgb(51,51,51);fo=
nt-family:Consolas,'Liberation Mono',Menlo,Courier,monospace;font-s=
ize:12px;line-height:16.8px;white-space:pre">);</span></span><br></div><br>=
<div>initializes the init_connection field to true by default, which was ki=
nd of surprising me, because it means that every new connection to an outpu=
t port with this policy will overwrite a sample previously written by anoth=
er port. Is this what we really want as a default? It is also not consisten=
t with the constructor called as</div><div><br></div><div><span class=3D"" =
style=3D"color:rgb(121,93,163);font-family:Consolas,'Liberation Mono=
9;,Menlo,Courier,monospace;font-size:12px;line-height:16.8px;white-space:pr=
e">ConnPolicy</span><span style=3D"color:rgb(51,51,51);font-family:Consolas=
,'Liberation Mono',Menlo,Courier,monospace;font-size:12px;line-heig=
ht:16.8px;white-space:pre;background-color:rgb(250,250,250)">(</span><span =
style=3D"color:rgb(51,51,51);font-family:Consolas,'Liberation Mono'=
,Menlo,Courier,monospace;font-size:12px;line-height:16.8px;white-space:pre;=
background-color:rgb(250,250,250)">DATA)</span><br></div><div><br></div><di=
v>which would initialize the init_connection flag to false. The static meth=
od data(...) is only used in the OutputPortInterface::createDataConnection(=
...) method from within RTT and OCL itself.</div><div><br></div><div>By the=
way, the init_connection flag is a good example where the API offers some =
degree of freedom to the application builder that semantically does not mak=
e sense at all for certain combinations, like in combination with a buffer =
connection. It is still not forbidden to set the flag, but it might be wort=
h a warning log message if a connection is created like this.</div><div><br=
></div><div><br></div><div><b>3) How to add the "shared" concept =
within the current API, or to use a "new" API?</b></div><div><br>=
</div><div>This point was brought up by Janosch, who asked for "<span =
style=3D"font-size:12.8px">a more explicit interface for shared connections=
", with the main argument that it is fundamentally different=C2=A0</sp=
an><span style=3D"font-size:12.8px">from a=C2=A0</span><span style=3D"font-=
size:12.8px">normal connection and a "connect port A to B with policy =
C" kind of API does not fit well to that concept. That's true. We =
considered to use the createStream(...) API first, with a name_id string th=
at identifies the shared buffer to connect to. This would actually fit bett=
er to the shared connection model, but it still enforces the same buffer ty=
pe policies for all connections. We withdraw that idea because the existing=
API is more general and disallowing certain policy combinations is require=
d anyway for private push connections with a single input buffer. At least =
in this case it cannot break existing applications.=C2=A0</span><span style=
=3D"font-size:12.8px">I do not think that there is a need for a completely =
new API only for shared connections.</span></div><div><span style=3D"font-s=
ize:12.8px"><br></span></div><div><span style=3D"font-size:12.8px">My curre=
nt implementation in the <a href=3D"https://github.com/orocos-toolchain/rtt=
/compare/master...updated-dataflow-semantics">updated-dataflow-semantics br=
anch</a>=C2=A0is as follows, and new/other ideas are always welcome:</span>=
</div><div><span style=3D"font-size:12.8px"><br></span></div><div><span sty=
le=3D"font-size:12.8px">- Shared connection objects (the shared data object=
or buffered) are added to a process-wide registry. Every shared connection=
is identified by a unique string, either set explictly in the name_id fiel=
d of the ConnPolicy or assigned automatically, in which case the name of th=
e connection is returned in exactly this field (name_id is and always was d=
eclared as mutable).</span></div><div><span style=3D"font-size:12.8px">- Fo=
r new connections with the name_id set, the shared connection is looked up =
in the registry, or created as new. As a consequence, if the same ConnPolic=
y instance is used to make a second connection, even if name_id was not set=
explicitly before, you will always connect to the same connection object.<=
/span></div><div><span style=3D"font-size:12.8px">- For new connections wit=
h no name_id set, the port objects provide an API to return a pointer to an=
existing shared connection if they are already connected to an existing on=
e, and only the non-connected port will be added. So ports can be connected=
in any order to each other, as long as there is always one of the two port=
s already connected. Otherwise the name_id has to be set or a new connectio=
n instance would be created. Perhaps this one could be dropped because it i=
t too implicit and does not work very well for remote connections...</span>=
</div><div><span style=3D"font-size:12.8px">- If an existing connection was=
found, the buffer policies (type, size and locking policy) have to match.<=
/span></div><div><span style=3D"font-size:12.8px">- All ports will only sto=
re a single pointer to the shared connection instance in their connection l=
ist (in internal::ConnectionManager) and cannot know directly which other p=
orts are connected.</span></div><div><span style=3D"font-size:12.8px">- For=
remote input ports with a shared connection, only a proxy is created local=
ly and can be looked up in the shared connection repository. Other ports wi=
ll only connect to either the =C2=A0proxy or the real shared connection ins=
tance, but never build a new remote channel once the proxy has been created=
for the first remote connection (works in both directions).</span></div><d=
iv><span style=3D"font-size:12.8px">- All connect and disconnect calls to s=
hared connections are logged with log level Debug to the RTT logger, so at =
least there is a manual way to check whether the resulting connections have=
been made as expected. Graphical tools like the <a href=3D"https://github.=
com/orocos-toolchain/rtt_dot_service">rtt_dot_service</a>=C2=A0could be pat=
ched later to visualize shared connections correctly.</span></div><div><spa=
n style=3D"font-size:12.8px"><br></span></div><div><br></div><div><b>4) Whe=
ther to encode the result of a write() in existing enums or not?</b></div><=
div><br></div><div>Janosch asked for introducing a separate enum for the re=
turn value of write(...) calls, in order to not break or trigger compiler w=
arnings for existing code that already uses the FlowStatus enum in a switch=
statement. Another minor issue with a single enum is the default value of =
FlowStatus attributes, which can be only meaningful for one use case.</div>=
<div><br></div><div>That's also fine for me and I will update the docum=
ent and implementation accordingly (add a new enum WriteStatus). Note that =
the evaluation in boolean context is counter-intuitive in this case, other =
than for FlowStatus, because the value WriteSuccess (=3D0) value evaluates =
to false and all error cases (WriteFailure and NotConnected) evaluate to tr=
ue.</div><div><br></div><div><br></div><div>I hope I did not forget anythin=
g. This mail already became much longer than expected. I am sorry.</div><di=
v><br></div><div>I also did some performance measurements of read and write=
calls last week in terms of absolute time, CPU time, number of assignments=
and so on of the current, the new and even the v1 data flow implementation=
s in different scenarios. I plan to come up with some results during this w=
eek. The tests revealed some bugs and performance caveats, and some of them=
are already present since the early days of RTT v2, e.g. that data samples=
might be lost (never read as NewData) because the read flag is not lock-pr=
otected or stored within the lock-free data structure. I am preparing a pul=
l request to the current master branch, but it will also break the API of c=
lass base::DataObjectInterface.</div><div><br></div><div>Best regards,</div=
><div>Johannes<br></div><div><br></div><div><br></div><div><div class=3D"gm=
ail_extra"><br><div class=3D"gmail_quote">On Mon, Sep 28, 2015 at 3:36 PM, =
S Roderick <span dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=
=3D"_blank">[email protected]</a>></span> wrote:</div><div class=3D"gmail=
_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex=
;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style=
:solid;padding-left:1ex"><div><div><br></div><div>Johannes' proposed ch=
anges are focussed on</div><div>a) push vs pull, or specifying whether the =
data/buffer object is on the reader or writer side. This is very useful for=
system designers dealing with significant time delay.=C2=A0</div><div>b) p=
rivate vs shared, whether a connection is per input/output port pair or whe=
ther the connection is shared between multiple input/output ports.</div><di=
v>c) mandatory, primarily intended to support knowing whether writes to a b=
uffer succeeded (so that traditional producer/consumer relationships have s=
ome knowledge about the buffer state)</div><div><br></div><div>It appears t=
o me that most people are worried about changing the defaults, more than th=
ey're worried about the actual proposed changes. Actual technical conce=
rns with the proposed changes appear to be</div><div><br></div><div>1) whet=
her to encode the result of a write() in existing enums or not</div><div>2)=
how to add the "shared" concept within the current API, or to us=
e a "new" API</div><div>3) changing the RTT defaults, which then =
affect an application's semantics.</div><div><br></div><div>Did I miss =
any technical issues?</div><div><br></div><div><br></div><div><div>My huge =
personal caveat to all of this is that I do *not* think we should change th=
e current system semantics in introducing this change. Too many Orocos chan=
ges have broken backwards compatibility and forced system designers and imp=
lementors to change their system in response. I don't think we should f=
orce that on users without an incredibly good reason. Otherwise it breeds f=
rustration in the community.</div><div><br></div><div>If there is some way =
to introduce the proposed changes to the system, but to not change the beha=
vior of current systems, then I think that is a win for all. There does see=
m to be some general agreement that there are valid use cases that the curr=
ent port implementation handles poorly, and that this proposed approach doe=
s fix some of those. The proposed approach also improves the situation for =
systems that don't want or need to use the flow status approach. Both a=
re equally valid approaches - IMHO it would be beneficial to the community =
as a whole if RTT supported both approaches.</div></div><div><br></div><div=
>Cheers</div><div>Stephen</div><div><blockquote type=3D"cite"><div><div><bl=
ockquote type=3D"cite"><br></blockquote></div></div></blockquote></div></di=
v><br>--<br>
Orocos-Dev mailing list<br>
<a href=3D"mailto:[email protected]">[email protected].=
kuleuven.be</a><br>
<a href=3D"http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev" rel=
=3D"noreferrer" target=3D"_blank">http://lists.mech.kuleuven.be/mailman/lis=
tinfo/orocos-dev</a><br>
<br></blockquote></div><div class=3D"gmail_signature"><div dir=3D"ltr"><spa=
n><font color=3D"#888888"><br></font></span></div></div>
</div></div></div>
--001a11349cee5e621e0520d409d9--
--===============1511504056==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--
Orocos-Dev mailing list
[email protected]
http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
--===============1511504056==--