Re: Traffic throttling

Thanks for the information. Honestly I didn't expect Pyro to be the
pinnacle of network performance; however, it is quite easy to use, and so
as long as it works I'm sticking with it. :)

I'll take a look at the thread limits, but I don't expect them to be a
problem since the computer generating the data is singlethreaded, and the
computer consuming it is trivially able to keep up (i.e. even if it does
spawn a thread for each new image, those threads don't live long).

-Chris


On Mon, Aug 19, 2013 at 4:14 PM, Irmen de Jong <[email protected]> wrote:

> On 19-8-2013 22:21, Derakon wrote:
> > I'm trying to fix a weird problem I have with sending large amounts of
> data between two
> > computers using Pyro4. One of the computers is generating images
> (540x512 Numpy arrays
> > of shorts, about .5MB each), and each one is sent to the other computer
> using a distinct
> > function call (i.e. one image/call). What I'm seeing is that we can
> sustain a rate of
> > about 550Mbit/sec, but randomly the network traffic between the two
> computers gets
> > throttled down to about 150Mbit/sec instead. Then a bit after that,
> traffic drops to
> > zero (the data source computer sees an RPC take >10s to return), before
> resuming at
> > 550Mbit/sec.
> >
> > I'm wondering if Pyro has any built-in throttling that might explain
> this issue.
>
> Not as such. It does have a (configurable) limit on the amount of threads
> being used in
> its threadpool-based socket server.  You also might want to try switching
> to the
> multiplex-server and see if that changes matters.
>
> Note that Pyro's wire protocol is not very efficient when dealing with
> large amounts of
> binary data. See
> http://pythonhosted.org/Pyro4/tipstricks.html#binary-data-transfer
>
>
> Irmen
>
>
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Pyro-core mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/pyro-core
>

------------------------------------------------------------------------------
Introducing Performance Central, a new site from SourceForge and 
AppDynamics. Performance Central is your source for news, insights, 
analysis and resources for efficient Application Performance Management. 
Visit us today!
http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk

_______________________________________________
Pyro-core mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pyro-core