[ZCM] [ZC] 2357/ 4 Comment "filestream_iterator and HTTP Range support"

Issue #2357 Update (Comment) "filestream_iterator and HTTP Range support"
 Status Pending, Zope/feature medium
To followup, visit:
  http://www.zope.org/Collectors/Zope/2357

==============================================================
= Comment - Entry #4 by mj on Sep 11, 2007 11:55 pm

Right, I indeed meant the interface, which we have used elsewhere to return other iterables to the publisher, so a derived interface would be preferable.

As for implementation, as the iterator provides only the body of the request, this would be very simple. You'd give it the range parameters, and the iterator would just spit out the right delimiters and data. See the OFS.Image and ZPublisher.HTTPRangeSupport modules for example code that parses range requests into usable datasets for this.

The application that returns this range iterator is then still responsible for setting the correct response headers and deal with conditional range requests, invalid range requests, unsatisfiable ranges and such. See the OFS.Image module for examples of such handling.
________________________________________
= Comment - Entry #3 by philikon on Sep 11, 2007 1:40 pm

Right now, ZPublisher.Iterators.IStreamIterator (which still is a Zope 2 interface, by the way, ick) does not specify seek()ing. I guess that's what mj is talking about.

He's wrong about the actual nature of the 'filestream_iterator' implementation, though. Because it *is* a file. In fact, it's a subclass of the 'file' type. It has to be a file on the filesystem in order to be efficient: if you want to stream lots of binary data out to the client without blocking lots of RAM, you want to let go of all application resources (in other words, ZODB objects). That's why you write the data to the filesystem and close the ZODB connection. The ZPublisher will then stream the stuff out to the client. IStreamIterator explains that.

Here's what we could do: We introduce an IRangeStreamIterator interface which extends IStreamIterator and adds a seek() or range() method or whatever is needed to support the Range use case. I bet 'filestream_iterator' could easily be extended to support it, since it's based on 'file'.
________________________________________
= Comment - Entry #2 by mj on Sep 6, 2007 12:29 pm

This would require seek access to the stream, which the iterator explicitly does not have.

This is a hard requirement, especially for Acrobat Reader, as it expects the ranges it requests to be returned in the same order, and it requests chunks ordered differently from actual file location.

The only workaround is to then cache the data for ranges in memory or a temp file, which becomes much more of an engineering undertaking. 
________________________________________
= Request - Entry #1 by ingeniweb2 on Sep 6, 2007 6:05 am

We cannot use the filestream_iterator "as is" when the HTTP client provides a "Range" header (Acrobat Reader does it) , because the fsi always provides the whole file content.

Would be nice for this to have a "range" parameter in the filestream_iterator constructor.

==============================================================