Re: Split bulk transfers

Hi Alan,

thanks for your clear answer. A few questions follow.

On Wednesday 09 May 2007, Alan Stern wrote:
> On Wed, 9 May 2007, Laurent Pinchart wrote:
> > Hi everybody,
> >
> > I'm running into an issue where large bulk transfers require lots of host
> > memory. To fix the problem (without upgrading the system memory), I tried
> > to implement "split bulk transfers". The first test results are
> > promising, but there are a few borderline cases that I'm not able to
> > test. I'd like to have your opinion on those. First of all, a little
> > background (I'll be talking about IN bulk transfers here) to make sure I
> > got everything right.
> >
> > Bulk transfers are payload oriented. The device usually reports the
> > maximum payload size, and the host allocated URBs big enough to transfer
> > a complete payload. The URBs are then divided in packets (by the host USB
> > stack or the host controller, I'm not sure about this), and data is
> > transfered packet by packet until the device signals the end of the
> > payload by a packet shorter than the maximum packet size, which can lead
> > to a 0-byte packet if the payload size is an exact multiple of the packet
> > size. The host detects that short packet, finishes processing the URB and
> > calls the completion handler. The short packet is required to detect
> > payload end, as the actual payload size might be smaller than the maximum
> > payload size reported by the device (payloads can have variable sizes).
> >
> > This works fine, but requires a lot of memory when the maximum payload
> > size is big. Someone reported running across a device with a maximum
> > payload size of 2.7MB, and as my driver allocates 5 bulk URBs, that makes
> > a lot of memory especially for embedded or other memory constrained
> > hosts. To work around this, the idea was to use smaller URBs that would
> > not carry a complete payload, but part of a payload. A payload would thus
> > be divided across several URBs, which would in turn be divided in USB
> > packets.
>
> The scatter-gather routines in drivers/usb/core/message.c do something
> similar to what you're describing.  But they assume that all the memory
> buffers have already been allocated, so they won't help solve your main
> problem (memory usage).

I wasn't aware of those functions. Just to make sure I get this right, am I 
right to say that URB buffers should be allocated as contiguous cache 
coherent physical memory (using usb_buffer_alloc, which in turns call 
dma_alloc_coherent for big DMAable buffers) or as normal memory using 
scatter-gather routines ?

I currently allocate big bulk buffers using usb_buffer_alloc, which puts a lot 
of pressure on the memory allocator. Using scatter-gather mappings would 
help. However, if I'm not mistaken, the usb_sg_* routines are designed for 
synchronous use (with usb_sg_wait), and thus can't be used for streaming 
data.

> > The problem I now have is to make sure I can detect the payload end
> > reliably. When the payload size is not an exact multiple of the URB size,
> > all the URBs except the last one will be filled with data. The last URB
> > will have an actual size smaller than its maximum size, which is how I
> > detect the payload end. When the payload size is an exact multiple of the
> > URB size, all URBs will have the same actual length. I would like to know
> > (and I can't test this as my test device has fixed-size payloads with
> > sizes that are not an exact multiple of the bulk packet size) if the USB
> > stack will return an empty URB to the completion handler, much like the
> > USB device sends an empty packet.
>
> Yes, it will.

Great.

> > If so, are there any other caveats I should be aware of when splitting the
> > payload across multiple bulk URBs ? And if no empty URB is currently
> > returned by the Linux USB stack, is that something that could/should/will
> > be fixed ?
>
> I assume that you submit your URBs one at a time, waiting for each one to
> complete before submitting the next, since the idea is to minimize memory
> usage.  If you were to submit several URBs at once, you would need to set
> the URB_SHORT_NOT_OK flag transfer flag on all but the last -- just
> theoretical, since you're not in this situation.

Actually I'm not. I submit a few URBs to get high throughput. Even with 
several URBs, memory usage is smaller than with a single payload-sized URB.

Why do I need to set URB_SHORT_NOT_OK on all but the last URB ? As the actual 
payload size isn't known (only the maximum payload size is known), a short 
packet can be received at any time. The next URBs will be filled with the 
next payload, right ?

> You should make sure that each URB's transfer_buffer_length is a multiple
> of the maxpacket size.  This is probably obvious; if you didn't do it then
> you would get an EOVERFLOW error.  There aren't any other constraints on
> the transfer sizes; you can make them as big or as small as you see fit.

That was indeed obvious, but still worth mentioning.

> There's a possible ambiguity if the maximum payload size is a multiple of
> the maxpacket size. If it is, and if the device wants to send a
> maximum-sized payload, does it also send a 0-length packet at the end?

That's a very good question. I haven't come across devices with a maximum 
payload size multiple of the maximum packet size.

> In theory it should not, but devices don't always obey the spec in this
> regard. Anyway, you will need to cope with the device's behavior.

I didn't know about that. Thanks.

> If it doesn't send the extra 0-length packet, then you have to make sure
> that your last URB's length is exactly the right amount to make up a
> maximum-sized payload.

Otherwise the last URB will be filled with packet from the next payload, 
right ?

> If it does send an extra 0-length packet then your last URB needs a larger
> length, so that the 0-length packet will be received properly.

If the last URB doesn't have a larger length, won't the 0-length packet be 
received by the next URB, making the USB stack call the completion handler 
with a 0-length URB ?

> Other than that, you should be okay.

Thanks a lot for your help once again.

Laurent Pinchart

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
[email protected]
To unsubscribe, use the last form field at:
https://lists.sourceforge.net/lists/listinfo/linux-usb-devel