Re: PPTPD upload speed bottleneck

On Sun, Nov 17, 2013 at 02:48:58PM -0000, Gustavo Homem wrote:
> The delay on pings, probably due to queueing, seems to be introduced
> between the eth and the ppp layer at the server. So it must be the
> server throttling the bandwidth and letting a queue of packets
> build, instead of something in the network.

I think you've certainly found a problem, but let's look at what could
be responsible.  Focusing on the arriving GRE packet on the server:

The packet is passed from the ethernet driver to the networking layer.

There are two processes to be notified, tcpdump and pptpctrl.  Both
processes have to be scheduled for execution.  pptpctrl has a raw
socket open to receive the packet.  pptp_gre_init.

pptpctrl will be in a select(2) call at the time, or it will be busy
processing a control connection packet or a previous GRE packet.  If
it is busy, there will be a delay before it gets around the event loop
to read(2) from the raw socket.  The delay could be something as
simple as a write(2) waiting for kernel resources.  It would vary each
time.

pptpctrl will call read(2) in decaps_gre to pull the packet up from
the raw socket, then it will strip the GRE encapsulation, check the
sequence number, and will encapsulate in HDLC and send it down to the
PTY for kernel PPP to handle.

Exactly when it will send it down will depend on the sequence number
in relation to previous sequence numbers.  If the sequence number is
the next expected, the packet will be sent down immediately.  If not,
there will be a delay of _at least_ 300ms.  This delay is of the same
order of magnitude as what you observe.

Some suggestions:

- look at the packet counters maintained by pptpd.  Any rx_buffered
  may represent a 300ms delay injection.

- try changing the DEFAULT_PACKET_TIMEOUT in pqueue.h from 0.3 to 0.1
  and see if the problem is changed.

- try commenting out the first "if (pptpctrl_debug)" in dequeue_gre
  and watch the logs.

Looking at what you pasted, I cannot find evidence of client to server
GRE packets with sequence numbers 534403 and 534404.  Line 77 of your
paste starts with 534405.  Packet 534403 is important because it will
tell me whether the delay will be triggered.

Another suggestion; do an strace with precise timings (-tt), and focus
on the select(2) in pptp_handle_ctrl_connection pptpctrl.c or the
read(2) in decaps_gre pptpgre.c.

Lastly, although I've looked at the code just now, I can't see how it
goes back to process the out of order packet queue in the absence of
network activity.  Perhaps you are hitting that.  It looks like a
bug.  Needs more eyes.

How does OpenVPN perform?  ;-)

-- 
James Cameron
http://quozl.linux.org.au/

------------------------------------------------------------------------------
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk