Re: zip64 format not streamable?

> I was going to write an example unzipper and started to look at the new
> zip64 format.  My intent was to write a streaming unzipper, i.e. one
> that does not seek in the zip file.  Reading PKWare's appnote leaves
> something indeterminate.  If a streaming zipper writes a local header
> with no length or crc information, setting bit 3 of the flags in the
> local header, then the compressed data is followed with a data
> descriptor that has the lengths and the crc.  That has been the case
> for many years since we got them to put that in their appnote when
> deflate first came out.  However in the new appnote it says: "for Zip64
> format archives, the compressed and uncompressed sizes are 8 bytes
> each" in that data descriptor.  However when bit 3 is set nothing
> preceding that will provide any indication that this is a "Zip64 format
> archive".  So is it four byte or eight byte lengths?

Reading the appnote (version 5.2, the first one I found) it seems in
that case the size fields in the local header could be set to 0xFFFFFFFF
(Zip64 flag value) and if bit 3 is set the actual sizes put in the
following descriptor.  That is not what the standard says though as when
bit 3 is set these need to be zero in the local header.

Could also detect the existence of a Zip64 extra field (could have all
zeros) and if exists and bit 3 set then data descriptor follows and is
Zip64.  The appnote does not mention that either but it follows.

Could also depend on the central directory header for sizes.  This makes
reading a stream difficult though.

As far as I can tell PKZip does not ever use this bit.  All their
archives seem to use temporary files to create multiple volume archives
then go back and set the local header fields.  PKZip apparently can read
archives with this bit set though as long as the right format is used
(including a data descriptor signature).  As far as I can tell anyway.

> Also this seems to go against the other instances of the zip64 format,
> where a zip64 extra field only appears when needed, in which case the
> old length fields are set to all ones to indicate the presence of the
> extra field.  But when bit 3 is set, the length fields are set to zero.

They could have handled or described this better.  I have been working
with this lately and hope to use bit 3 shortly.

> So has PKWare screwed up the format so that it isn't streamable, or am
> I reading something wrong?  Or maybe if the version is > 45, it's a
> "Zip64 format archive", and the data descriptors always have eight-byte
> lengths?

I think the data descriptor changes size as needed for each file.  They
may not have actually tested this mode.

By the way, you may want to also mail Zip-Bugs.

Ed Gordon