Re: High SFTP CPU usage

Alexander V. Lukyanov wrote:
> On Thu, May 20, 2004 at 12:06:26PM -0700, Andy Lutomirski wrote:
> 
>>Disclaimer: these results are patently absurd.  So I'll retest later.
>>
>>The fast server that I've been testing against is down, so I tested 
>>against localhost.  I did a ~/test3/bin/lftp -c 'open sftp://localhost; 
>>get 040331.rm' (from a reiserfs partition to an XFS partition on another 
>>physical disk).  Both of these disks can sustain 40-60MBps.
>>
>>I got a whopping 1.45MBps.  lftp used ~46% CPU.  ssh and sshd each used 
> 
> 
> Thanks for testing!
> 
> I have found and fixed a bug which caused extra spinning. Please test rc4.
> 

I'm still just testing over localhost.

get is up to 2.5MBps or so, and CPU in that case is down to less than 
2%.  There's a decent amount of time spent in iowait, but the CPU is 
also about half idle.  (This is still much better than rc3).

Strangely, this works very quickly:

[andy@luto tmp]$ time /home/andy/test4/bin/lftp -c 'set 
sftp:connect-program /usr/libexec/openssh/sftp-server; open 
sftp://localhost; get /home/andy/testfile'

real    0m1.010s
user    0m0.372s
sys     0m0.405s

(testfile is 105MB)

So I guess there's something funny going up with lftp's interaction with 
sftp.

Uploads (using real ssh) are more-or-less unchanged, and lftp is using 
~~11% CPU.

Is it possible that lftp is reading very small pieces, which does 
something bad?

I'm not actually sure any of this is worth optimizing for, because lftp 
over localhost is somewhat absurd.  I'll email you once the server I use 
comes back up.

Thanks,
Andy