Re: Header folding

At 8/19/2004 04:33 AM, Mark Overmeer wrote:
>In human translation, this says: fold the unfolded version of the body
>when either a wrap-length is provided (in @_) or the line is not folded
>yet (isn't terminated by a \n).
>
>Where I try to be strict on, is that there are two different ways you
>can have the body (the contents) of a field in:
>   1) unfolded form: no new-lines at all
>   2) folded form: newline after each line.

I think I wasn't clear in my explanation, as I agree with your points about 
folding (processing power, clarity, etc.).  The issues is really with 
Mail::Box::Parser::Perl.  Even if a header has only a single line, the 
parser leaves the \n at the end of it.  In fact, the parser even sticks in 
a single \n if the parse failed and the body of the header is empty.  The 
implication of this is that the parser effectively says that any parsed 
header is in a "folded" form -- even if it's a single line -- and thus 
setWrapLength() will always refuse to wrap it to the default length.  It 
will only wrap if given a specific length to wrap to.

 From what I can see, this appears to be different from the way a header 
body gets added normally with add() -- I didn't see any attempt to ensure 
that a trailing \n be added.  (I could be wrong, of course.)

The logic I had of removing a trailing \n in parsing, but keeping \n in the 
middle of a body is that it helps disambiguates case #2 above.  A body is 
folded if there are *any* newlines in it.  With the parser change and the 
setWrapLength change, a single-line parsed field gets wrapped if it's over 
the default length.  I'm not sure where the trailing \n gets added during 
printing/stringification -- I got lost trying to follow through all the 
calls -- but the patches I made passed the test suite, so I figured it was 
handled somewhere (i.e. stuck on if it wasn't already there).

>For instance:
>
>  X-Postal: The White House
>   1600 Pennsylvania Avenue NW
>   Washington, DC 20500
>
>The unfolded version will be:
>   "The White House 1600 Pennsylvania Avenue NW Washington, DC 20500"
>
>The folder version will be equivalent to
>     " The White House\n"
>   . " 1600 Pennsylvania Avenue NW\n"
>   . " Washington, DC 20500\n"
>
>No intermediate versions of the field content should exist.

This actually illustrates the issue.  You have no trailing \n on the first 
version, but you do on the second.  So either the unfolded should be:

   "The White House 1600 Pennsylvania Avenue NW Washington, DC 20500\n"

or the second version should be:

     " The White House\n"
   . " 1600 Pennsylvania Avenue NW\n"
   . " Washington, DC 20500"

I'd prefer consistency one way or the other.

>One big issue with your proposed solution (removing the last \n
>everywhere) is that a line me be ending on \r\n  :-(  and that
>should be honoured.

Hmm. I could be mistaken, but I seem to recall skimming past some code that 
explicitly looked for \012\015 and changed it to \n.  I think it's 
consume() which looks for "erroneous wrap separators"?

>So... I do not see a bug (be that can be my limited view on the world)
>and therefore no reason to fix things.  Tell me if I'm wrong.

You're never wrong!  You only have a different opinion sometimes.  :-)

Frankly, I'm not sure what the right approach is.  I like the proposed 
wrap() function.  I think the line ending should be consistent for both 
folded and unfolded cases unless there's a strong reason not to do so.  And 
either way, I think it could use some clarification in the 
documentation.  Definitely, if the notion that "parsed fields are always 
taken verbatim as being wrapped and will not be re-wrapped unless wrap() is 
explicitly called by the user" is the outcome.

Regards,
David