Re: [jgroups-users] RPC request and response message prioritization

On 21/06/16 01:37, Questions/problems related to using JGroups wrote:
> Hello Bela, et al.,
>
> Background first:
>
> We must prioritize our messages using PrioHeader and RATE_LIMITER2.  To
> prioritize messages, we add a PrioHeader to the message at the
> application level.  We have combined the PRIO and RATE_LIMITER2 into a
> new version of RATE_LIMITER2 – we are no longer are using PRIO.

Can I remove PRIO in master then?

> RATE_LIMITER2 now contains a priority queue to manage the message
> prioritization.  We are using two instances of RATE_LIMITER2 in the
> protocol stack, one for prioritized messages (from our application) and
> one for non-prioritized messages (mostly JGroups messages).   Message
> either have or do not have a PrioHeader and thus depending on this
> logic, messages are “in scope” for only one of the RATE_LIMITER2 instances.
>
> Problem:
>
> We use RPC’s to send requests to get large data sets.  We use NEAREST to
> determine the nearest network host from which we will make the data
> request to optimize the network usage.  I was easily able to modify the
> RpcDispatcher to add a PrioHeader to the RPC request messages so that
> the request flows properly through the rate limited stack by adding a
> priority to the RequestOptions that is an argument to the
> callRemoteMethod method and then from the RequestOptions extract a
> priority to add to the message.  This functionality by default is
> disabled and can be enabled with a function call to the RpcDispatcher.


OK. I'm reading on without knowing what PRIO, RATE_LIMITER2 or NEAREST 
are doing... :-)


> The problem I have is prioritizing the RPC method response which, as you
> can image, is where the large amounts of data needs to be transferred.
> The problem is that the RequestCorrelator, after calling the
> RpcDispatcher.handle method, creates the response messages with a fixed
> set of headers and thus it has removed the PrioHeader that I added in
> the rsp instance in RpcDispatcher.handle.

Wait: handle() returns an Object not a Message so how can you add 
headers there?

If you extended RequestCorrelator.sendReply() and set your subclass in 
RpcDispatcher, would that help? Or you could override sendResponse() or 
prepareResponse().

> Here’s the code from the RequestCorrelator
>
>
> protected void sendResponse(Message rsp, long req_id, boolean is_exception) {
>      prepareResponse(rsp);
> Header rsp_hdr=new Header(is_exception? Header./EXC_RSP /: Header./RSP/,
> req_id, corr_id);
> rsp.putHeader(corr_id, rsp_hdr);
>      if(/log/.isTraceEnabled())
> /log/.trace("sending rsp for %d to %s", req_id, rsp.getDest());
> transport.down(new Event(Event./MSG/, rsp));
> }


> My changes thus far (outside of changes to RATE_LIMITER2) have been
> “additional” functionality that would not impact other JGroups users.
> However, when I think about what I would need to do to add a PrioHeader
> to an RFC response, it makes me pause and reach out to you to see if you
> have a better solution or another suggestion.

I could add a callback to MessageDispatcher/RpcDispatcher that let's you 
take a look at the request and create a response accordingly, e.g. by 
copying selected flags/headers from the request into the response.

Or would subclassing RequestCorrelator and setting it in RpcDispatcher 
work for you?


> BTW:  We had a similar issue with FRAG2 in that it only preserves the
> headers for the first fragment – which makes perfect sense – however,
> the removal of the PrioHeader results in all but the first fragment to
> be directed to the non-prioritized instance of RATE_LIMITER2.  I
> modified FRAG2 slightly to add a new property “preserve_headers” which
> is false by default.  If it is set to true, all message fragments will
> contain all of the original headers. Very simple, very clean.
>
> Thanks for your time.
>
> Thanks,
>
> **
>
> *Michael Earl *
>
> Software Engineering Manager
>
> All Weather Inc.
>
> 1165 National Drive
>
> Sacramento, CA 95834
>
> E-mail: mailto:[email protected]


-- 
Bela Ban, JGroups lead (http://www.jgroups.org)


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape