Re: [jgroups-users] RPC request and response message prioritization

Hi Bela,

See answers below.


Thanks,

Mike.


-----Original Message-----
From: Questions/problems related to using JGroups [mailto:[email protected]] 
Sent: Tuesday, June 21, 2016 12:05 AM
To: [email protected]
Subject: Re: [jgroups-users] RPC request and response message prioritization



On 21/06/16 01:37, Questions/problems related to using JGroups wrote:
> Hello Bela, et al.,
>
> Background first:
>
> We must prioritize our messages using PrioHeader and RATE_LIMITER2.  
> To prioritize messages, we add a PrioHeader to the message at the 
> application level.  We have combined the PRIO and RATE_LIMITER2 into a 
> new version of RATE_LIMITER2 - we are no longer are using PRIO.

Can I remove PRIO in master then?
[ME] Yes

> RATE_LIMITER2 now contains a priority queue to manage the message 
> prioritization.  We are using two instances of RATE_LIMITER2 in the 
> protocol stack, one for prioritized messages (from our application) and
> one for non-prioritized messages (mostly JGroups messages).   Message
> either have or do not have a PrioHeader and thus depending on this 
> logic, messages are "in scope" for only one of the RATE_LIMITER2 instances.
>
> Problem:
>
> We use RPC's to send requests to get large data sets.  We use NEAREST 
> to determine the nearest network host from which we will make the data 
> request to optimize the network usage.  I was easily able to modify 
> the RpcDispatcher to add a PrioHeader to the RPC request messages so 
> that the request flows properly through the rate limited stack by 
> adding a priority to the RequestOptions that is an argument to the 
> callRemoteMethod method and then from the RequestOptions extract a 
> priority to add to the message.  This functionality by default is 
> disabled and can be enabled with a function call to the RpcDispatcher.


OK. I'm reading on without knowing what PRIO, RATE_LIMITER2 or NEAREST are doing... :-)


> The problem I have is prioritizing the RPC method response which, as you
> can image, is where the large amounts of data needs to be transferred.
> The problem is that the RequestCorrelator, after calling the
> RpcDispatcher.handle method, creates the response messages with a fixed
> set of headers and thus it has removed the PrioHeader that I added in
> the rsp instance in RpcDispatcher.handle.

Wait: handle() returns an Object not a Message so how can you add 
headers there?
[ME] Handles argument is a Message
@Override
    public Object handle( Message req) throws Exception {
I add the PrioHeader to the req argument 

If you extended RequestCorrelator.sendReply() and set your subclass in 
RpcDispatcher, would that help? Or you could override sendResponse() or 
prepareResponse().
[ME] Good Idea.  I extended RpcDispatcher and RequestCorrelator and override the necessary methods (1 method in each class) and it worked fine.  Thanks!

> Here's the code from the RequestCorrelator
>
>
> protected void sendResponse(Message rsp, long req_id, boolean is_exception) {
>      prepareResponse(rsp);
> Header rsp_hdr=new Header(is_exception? Header./EXC_RSP /: Header./RSP/,
> req_id, corr_id);
> rsp.putHeader(corr_id, rsp_hdr);
>      if(/log/.isTraceEnabled())
> /log/.trace("sending rsp for %d to %s", req_id, rsp.getDest());
> transport.down(new Event(Event./MSG/, rsp));
> }


> My changes thus far (outside of changes to RATE_LIMITER2) have been
> "additional" functionality that would not impact other JGroups users.
> However, when I think about what I would need to do to add a PrioHeader
> to an RFC response, it makes me pause and reach out to you to see if you
> have a better solution or another suggestion.

I could add a callback to MessageDispatcher/RpcDispatcher that let's you 
take a look at the request and create a response accordingly, e.g. by 
copying selected flags/headers from the request into the response.

Or would subclassing RequestCorrelator and setting it in RpcDispatcher 
work for you?

[ME] --- As mentioned above, extending those two classes was perfect!


> BTW:  We had a similar issue with FRAG2 in that it only preserves the
> headers for the first fragment - which makes perfect sense - however,
> the removal of the PrioHeader results in all but the first fragment to
> be directed to the non-prioritized instance of RATE_LIMITER2.  I
> modified FRAG2 slightly to add a new property "preserve_headers" which
> is false by default.  If it is set to true, all message fragments will
> contain all of the original headers. Very simple, very clean.
>
> Thanks for your time.
>
> Thanks,
>
> **
>
> *Michael Earl *
>
> Software Engineering Manager
>
> All Weather Inc.
>
> 1165 National Drive
>
> Sacramento, CA 95834
>
> E-mail: mailto:[email protected]


-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
javagroups-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/javagroups-users

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape