Re: Handling text files

Due to some oppositions, I am withdrawing my proposal to modify the
behavior of "unzip -a" so that CR CR LF is considered as one line
instead of two.

However, I still maintain my suggestion about the new strategy to
set the "binary/text" flag in zip, and about not transforming CR LF
sequences into CR CR LF when using "zip -l".


By the way: I noticed that I forgot about the Ctrl-Z character, with
the code 26. It appears as EOF on many platforms.

>In set_file_type() from trees.c, you set the file type as follows:
>   set file type to ASCII
>   for n in { 0, 1, ..., 8, 14, ..., 31 } do
>     if dyn_tree[n].Freq != 0 then
>       set file type to BINARY
>     end if
>   end for

The corrected list is { 0, 1, ..., 8, 14, ..., 25, 27, ..., 31 }.


Greg wrote:

>Ah, slick.  But does the text/binary flag go only in the central header
>or also in the local?  Processing the whole file destroys streamability
>in the latter case (unless the extended local header can contain it,
>too, but I'm guessing not).  I should just check myself, but it's late
>and I'm lazy.

AFAIK, set_file_type() is called when the first deflate buffer is
flushed. So the method is streamable, but not entirely correct because
the rest of the file is overlooked. I still consider this "good enough",
though. Even by going over the entire file twice, you can never guess
with 100% accuracy.

> > MacOS up to v9 uses exclusively CR, doesn't it? Files that contain
> > "text\r\r\ntext\r\r\ntext\r\r\n" are unlikely to come from MacOS.
>
>Perhaps, but now you're making an additional assumption--the current
>conversion code sees, at most, one \r\r\n and decides on that basis.
>(In fact, I'm pretty sure it sees only \r\r before making a decision.)
>Doing a global check would require a two-pass approach and is unlikely
>to be a popular change.

You don't need two passes. You can look at txt2dos program that I used
when I converted zlib-1.2.1.tar.gz to zlib121.zip.

  http://www.cs.toronto.edu/~cosmin/prog/txt2dos.c
  http://www.cs.toronto.edu/~cosmin/prog/txt2ux.c

Although the programs make two passes, to identify first if the file is
binary, the actual text-to-text conversion is done on the fly. Look
inside the functions txt2dos() or txt2ux().

But this is moot, since I have withdrawn my proposal.


Best regards,
Cosmin

_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE*  
http://join.msn.com/?page=dept/bcomm&pgmarket=en-ca&RU=http%3a%2f%2fjoin.msn.com%2f%3fpage%3dmisc%2fspecialoffers%26pgmarket%3den-ca