Re: Handling text files

Darren Salt wrote:

> > That's not a bad idea, although the current approach was chosen for
> > performance reasons.  However, insofar as most Zip-running machines are
> > ridiculously overpowered these days, [...]
>
>*Definitely* not all (say I, using a 40MHz ARM710)...

It doesn't matter. My scheme is slightly _faster_ than what we currently
have.


Ed Gordon wrote:

> > I propose TO modify zip so that "zip -l" leaves CR LF intact.
>
>There may be users that depend on the LF to CR LF conversion just adding
>CR before each LF.

My proposal preserves the current behavior, if LF is not preceded by CR.
But it does not insert a CR if a LF is *already* preceded by a CR. What
users would be negatively affected by this?

>Changing zip -l would likely lead to incompatibility
>problems for some users and catch them by surprise.  Maybe a new option
>like -lc might convert LF CR LF to CR LF CR LF.  Changing defined
>options to act differently is I think generally a bad idea.

I would agree, if a normal user would find a normal use in the expansion
of CR LF to CR CR LF. Instead, I think that "zip -l" was intended to
be run on Unix text files, and the outcome of running it on DOS text
files is just "whatever happens". And it happens to do something that is
hardly useful, and it's hard to believe that someone relies on it.
In other words, I consider the current behavior "buggy".

Personal experience: the first time I used "zip -l", I was unpleasantly
surprised to see that some of my text files have been modified in a way
that I did not like. I had expected "zip -l" to act like "unzip -a".


> > My proposed method does not incur any performance penalty.
> > In set_file_type() from trees.c, you set the file type as follows:
> >   set file type to ASCII
> >   for n in { 0, 1, ..., 8, 14, ... 31 } do
> >     if dyn_tree[n].Freq != 0 then
> >       set file type to BINARY
> >     end if
> >   end for
>
>The above does sound reasonable, but there are a couple cases that might
>cause trouble.  Corrupted text files for one, where random characters
>get inserted by accident.  A single corrupted line could force binary
>using the above.

I think I understand what you mean, and yet I would prefer to sacrifice
a corrupted text file by not converting it, rather than sacrificing a
valid binary file by converting (I mean, corrupting) it.
If it's a corrupted text, someone should "fix" it first, and I believe
that the repairman knows enough to convert the line endings to whatever,
after the file is fixed.

>Another is special characters used as printable by one
>platform character set that are not printable on another.

The set that I proposed does not have to be hard-coded. It can be
redefined for every platform that uses funny symbols as printables.
Also see iscntrl().

>This should consider other languages.

All the language encodings I heard of have their letters above 127, not
below 31. UCS-2 and UCS-4 are of course exceptions, but the current zip
doesn't recognize them anyway.

As a matter of fact, the current zip is less language-tolerant than
my proposal. It rejects all texts that contain more than 1/4 symbols
in the range 128-255. Which means that people who write in Greek,
Cyrillic, Asian, etc., are out of luck.


Best regards,
Cosmin

_________________________________________________________________
The new MSN 8: advanced junk mail protection and 2 months FREE*  
http://join.msn.com/?page=dept/bcomm&pgmarket=en-ca&RU=http%3a%2f%2fjoin.msn.com%2f%3fpage%3dmisc%2fspecialoffers%26pgmarket%3den-ca