Re: [jgroups-dev] sharing channels in JGroups

Nuno Carvalho wrote:
> Hi,
>
> On Nov 6, 2010, at 3:41 PM, Bela Ban wrote:
>
>> Currently, the only possibility is to create 2 channels, but I 
>> suggest to use a shared transport, which contains all resources (e.g. 
>> default and OOB thread pool, timer, sockets etc).
>>
>> I already have the ability to bypass certain protocols, e.g. NO_FC 
>> set in a message bypasses any flow control layer. It would be easy to 
>> do this for SEQUENCER, too, but I want to come up with a generic 
>> approach to this.
>>
>
> This is exactly what we are doing at this moment. We have an 
> additional flag in the message to tell the sequencer protocol if it 
> needs to handle the message or not. I just wanted to know if there was 
> a way to do this without having to change the source code of jgroups.

I actually added 2 flags NO_RELIABILITY and NO_TOTAL_ORDER, and modified 
UNICAST, UNICAST2, NAKACK, SMACK and SEQUENCER to take this into 
account. I committed this ca 2 hours ago to CVS head.


> With two channels there is a big waste of resources.

It's not too bad, as all resources are pooled in the shared transport, 
and other protocols (except for FD_SOCK) don't have resources. But I do 
agree that a bypass flag is better.


> In the case of the NO_FC flag, how do you ensure FIFO order between 
> two messages m1(NO_FC == true) and m2(NO_FC==false)?

NAKACK (for multicasts) or UNICAST (for unicasts) take care of that. The 
flow control protocols (UFC, MFC) are on top of the reliability 
protocols. When sending  messages, the order in which they hit the 
reliability protocol assigns sequence numbers. On the receiver side, the 
messages are delivered in order of seqnos.


>> What I have in mind [1] is something similar to what we did in 
>> Ensemble: use *properties*, e.g. +reliable-mcast, -frag, 
>> +flowcontrol, -bundling etc etc etc. Based on these flags, each 
>> protocol decides whether to handle (or not) a message.
>>
>
> This sounds good.
>
> In Appia we solve this by having several channels, each one for 
> different needs in terms of message ordering, reliability, etc. 
> Protocols only accept a known (and previously defined) set of messages 
> types, so messages will bypass protocols and this is defined in 
> compilation time. But sometimes there is also a need for the 
> application to send some information in the message that is only 
> relevant to a certain protocol in the communication stack and this 
> scheme of properties fits very well.

Good. After implementing NO_RELIABILITY and NO_TOTAL_ORDER, I pushed [1] 
to 3.0 though, as these 2 flags are a workaround until I implement 
properties.

I'm focusing on RELAY in 2.12, and want to release 2.12 soon, hopefully 
before Feb, and then tackle 3.0.


>> This can probably be generalized, ie. by annotating a protocol with 
>> the props it provides, e.g. @Reliable @ReliableUnicast UNICAST, and 
>> by having code which matches the flags in a message and decides 
>> whether or not to process the message.
>>
>
> Where this code would run? Before delivering a message (or not) to 
> each protocol?

Yes, we'd run a generic check before each protocol's up() or down() 
method, but only for protocols which do have these annotations. The 
annotations would get converted into bit sets at initialization time, 
which can be compared to the message's bit set.

> I suppose that this could be done in a generic way if the code is in 
> the part that is responsible to deliver a message to the next protocol.

Yes, exactly

> Deciding to deliver a message to a protocol can be fast if it's just a 
> bit comparison between two bitmaps: one provided by the message and 
> another provided by the protocol with the properties that it 
> guarantees. The code could be something like:
>
> if( (message.bitset & protocol.bitset) != 0)
>     protocol.handle(message);


Exactly what I have in mind !

>
>>
>> [1] https://jira.jboss.org/browse/JGRP-1250
>>
>
> I've noticed that you added the NO_SEQUENCER flag in the message 
> (comment on JIRA entry). I suppose that we can find the code in the 
> CVS, right? Is the code in CVS head stable?

Yes, very stable. I just released 2.11 (a week ago) and haven't had much 
time to commit code to head, as I've been tackling the design of RELAY.

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss


------------------------------------------------------------------------------
The Next 800 Companies to Lead America's Growth: New Video Whitepaper
David G. Thomson, author of the best-selling book "Blueprint to a 
Billion" shares his insights and actions to help propel your 
business during the next growth cycle. Listen Now!
http://p.sf.net/sfu/SAP-dev2dev
_______________________________________________
Javagroups-development mailing list