Re: CIFS slow on gigabit, doesn't support sockopt=TCP_NODELAY ?

On Mon, Aug 24, 2009 at 3:42 PM, Jeff Layton<[email protected]> wrote:
> On Mon, 24 Aug 2009 14:36:37 -0400
> Timothy Normand Miller <[email protected]> wrote:
>
>> On Mon, Aug 24, 2009 at 9:50 AM, Jeff Layton<[email protected]> wrote:
>> > On Sun, 23 Aug 2009 20:21:10 -0400
>> > Timothy Normand Miller <[email protected]> wrote:
>> >
>> >> Hi, everyone.
>> >>
>> >> I've noticed that the cifs client for Linux is slow over gigabit
>> >> ethernet.  It seems to max out at about 10 megs/sec, while the drives
>> >> can go a lot faster.  Also, if I mount the same network share from a
>> >> Mac, it's a lot faster.  When _sharing_ via samba, one can set the
>> >> TCP_NODELAY option (among others), which fixes the problem.  But with
>> >> the cifs client, I find that there appears to be no way to set the
>> >> option.  When mounting manually, you can use "-o sockopt=TCP_NODELAY",
>> >> and you can also put that into /etc/fstab.  Either way, the option
>> >> appears to be ignored.
>> >>
>> >
>> > Yeah. The sockopt= option is completely ignored. I proposed a patch
>> > to remove it not too long ago...
>> >
>> > I'm unclear...so you're saying here that turning on TCP_NODELAY on the
>> > server it helps throughput when using the Linux CIFS client?
>>
>> No.  I think I'll give you excess detail just to make sure that I
>> don't leave out something important.  :)
>>
>> I have a Linux server, an Apple AEBS with disks hanging off of it, and
>> various clients (some Macs).  On the Linux server, I can configure the
>> SMB server to enable those options so that when a client accesses the
>> server, it's fast.  The AEBS is also a server, but I cannot configure
>> it with those options.  I can get a Mac to get fast access to the AEBS
>> by configuring those TCP options on the client side.
>>
>> The problem pertains to having the Linux box access the disks on the
>> AEBS.  In that case, I can't configure the TCP options on either the
>> client (the Linux box in this case) or the server (the AEBS), so
>> transfers are slow.  This is important, because a large part of my
>> large file copies are between the Linux box and the AEBS.
>>
>> On the Linux server, SMB configuration has no impact on CIFS performance.
>>
>
> If you wanted to experiment with it, it doesn't sound too hard to hack
> up a patch to do so. You'd just need to add a kernel_setsockopt call to
> the ipv4_connect (and/or ipv6_connect) functions. The place where the
> sk_rcvtimeo/sk_sndtimeo get set is probably around the right spot. In
> fact, those should probably be changed to kernel_setsockopt calls too.
>
> If that shows notable improvement then we could discuss the best way to
> set up an option for it. Probably we'd want to flesh out the sockopt=
> option.

I don't have much experience with Linux kernel drivers, so maybe you
can give me some hints.

(1) There's a userspace component (mount.cifs).  Is the problem in there?
(2) Or is it in the kernel driver?
(3) How are the options parsed?  Is this done for us?  That is, do I
need to look at existing data structures, or do I need to add some
parsing code?
(4) Is it okay to discuss this on this list, or should I move to a
different one?

Thanks.

-- 
Timothy Normand Miller
http://www.cse.ohio-state.edu/~millerti
Open Graphics Project