MIME multipart boundaries and S/MIME

This is a multi-part message in MIME format.
--------------070903000501090303060605
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I'm trying to fix the problem where Mail::Box breaks the signature of 
some S/MIME messages.  This requires fairly extensive changes to fix, so 
I'd like to get your input on how to design the fix.

The problem stems from the failure of Mail::Box to conform to the 
following paragraph in section 5.1.1 of RFC 2046:

   NOTE:  The CRLF preceding the boundary delimiter line is conceptually
   attached to the boundary so that it is possible to have a part that
   does not end with a CRLF (line  break).  Body parts that must be
   considered to end with line breaks, therefore, must have two CRLFs
   preceding the boundary delimiter line, the first of which is part of
   the preceding body part, and the second of which is part of the
   encapsulation boundary.


The most common problem is that 
Mail::Box::Parser::Perl::_read_stripped_lines will remove a blank line 
before a boundary delimiter line, but 
Mail::Message::Body::Multipart::lines will not put it back in.  
Mail::Message:Body::Multipart::lines is clearly wrong--in order to 
conform to the MIME standard, it needs to insert a CRLF before any 
boundary delimiter line preceeded by a body-part.

Mail::Box::Parser::Perl::_read_stripped_lines is also incorrect in only 
removing the CRLF before the boundary delimiter line when that CRLF is a 
blank line.  As quoted above and as can be deduced from the MIME 
grammar, it is possible to have a part that does not end with a CRLF.  
In the case where the line before a boundary delimiter line is not 
blank, the @lines array for the preceeding body needs to end with a line 
with no trailing CRLF.  This loosens the previously held invariant that 
every line in a @lines array ends with CRLF, allowing the last line in a 
@lines array to omit a CRLF.  This brings up the first design question: 
should the nrLines() method  include an unterminated final line in its 
returned count?  If so, then Mail::Message::Body::Multipart::nrLines has 
to peek at the last line of every subpart to see if it is unterminated.  
If not, then leaf implementations of nrLines() need to peek at their 
respective last lines to see if they are unterminated.

Complicating this is the issue of preambles.  Per the multipart grammar, 
if there is no preamble, then it is not necessary to have a CRLF before 
the first boundary delimiter line.  I can see two ways of addressing 
this: consider any CRLF to be part of the preamble or somehow encode the 
difference between no preamble (no CRLF before first boundary delimiter 
line) and a 0-length preamble (one CRLF before first boundary delimiter 
line).  Either choice requires an interface change to 
Mail::Box::Parser::Perl::_read_stripped_lines and its immediate callers 
(bodyAsString, bodyAsList, bodyAsFile, and bodyDelayed).  The first 
choice requires an additional argument saying whether or not a preamble 
is being read, the second choice requires a way to return the difference 
between "stripped CRLF" and "no CRLF to strip".

Attached is my work so far.  I'm making the parser treat MIME multipart 
boundaries different than mailbox boundaries.  For mailbox boundaries, 
the lack of a blank line before the separator will not cause the CRLF to 
be stripped from the preceeding line.  I appreciate opinions on which 
direction to take this in.



--------------070903000501090303060605
Content-Type: text/plain;
 name="mailbox-signed2.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="mailbox-signed2.diff"

Only in Mail-Box-2.060-5signed2: blib
diff -ur Mail-Box-2.060-4version/lib/Mail/Box/Parser/Perl.pm Mail-Box-2.060-5signed2/lib/Mail/Box/Parser/Perl.pm
--- Mail-Box-2.060-4version/lib/Mail/Box/Parser/Perl.pm	2005-04-01 10:54:55.000000000 -0800
+++ Mail-Box-2.060-5signed2/lib/Mail/Box/Parser/Perl.pm	2005-06-10 14:47:05.000000000 -0700
@@ -102,14 +102,18 @@
     # No seps, then when have to trust it.
     my $sep = $self->{MBPP_separators}[0];
     return 1 unless defined $sep;
+    my $ismimemultipart = $sep =~ /^--/;
 
     my $file = $self->{MBPP_file};
     my $here = $file->tell;
-    $file->seek($where, 0) or return 0;
+    $file->seek($$where, 0) or return 0;
 
     # Find first non-empty line on specified location.
     my $line = $file->getline;
-    $line    = $file->getline while defined $line && $line =~ $empty;
+    while (defined $line && $line =~ $empty) {
+	$$where = $file->tell if $ismimemultipart;
+	$line    = $file->getline;
+    }
 
     # Check completed, return to old spot.
     $file->seek($here, 0);
@@ -197,19 +201,9 @@
     {   $lines = ((ref $file eq 'Mail::Box::FastScalar') || (ref $file eq 'Proofpoint::IO::FastScalar')) ? $file->getlines : [ $file->getlines ];
     }
 
+    pop @$lines if (@seps && $seps[0] !~ /^--/ && @$lines && ${$lines}[-1] =~ $empty);
+
     my $end = $file->tell;
-    if($exp_lines > 0 )
-    {    while(@$lines > $exp_lines && $lines->[-1] =~ $empty)
-         {   $end -= length $lines->[-1];
-             pop @$lines;
-         }
-    }
-    elsif(@seps && @$lines && $lines->[-1] =~ $empty)
-    {   # blank line should be in place before a separator.  Only that
-        # line is removed.
-        $end -= length $lines->[-1];
-        pop @$lines;
-    }
 
     map { s/^\>(\>*From\s)/$1/ } @$lines
         if $self->{MBPP_strip_gt};
@@ -241,7 +235,7 @@
     {   # Get at once may be successful
         my $end = $begin + $exp_chars;
 
-        if($self->_is_good_end($end))
+        if($self->_is_good_end(\$end))
         {   my $body = $self->_take_scalar($begin, $end);
             $body =~ s/^\>(\>*From\s)/$1/gm if $self->{MBPP_strip_gt};
             return ($begin, $file->tell, $body);
@@ -286,7 +280,7 @@
     if(defined $exp_chars)
     {   my $end = $begin + $exp_chars;
 
-        if($self->_is_good_end($end))
+        if($self->_is_good_end(\$end))
         {   $file->seek($end, 0);
             return ($begin, $end, $exp_chars, $exp_lines);
         }
Only in Mail-Box-2.060-5signed2/lib/Mail/Box/Parser: Perl.pm~
diff -ur Mail-Box-2.060-4version/lib/Mail/Message/Body/Multipart.pm Mail-Box-2.060-5signed2/lib/Mail/Message/Body/Multipart.pm
--- Mail-Box-2.060-4version/lib/Mail/Message/Body/Multipart.pm	2005-04-01 10:54:56.000000000 -0800
+++ Mail-Box-2.060-5signed2/lib/Mail/Message/Body/Multipart.pm	2005-06-13 11:25:43.000000000 -0700
@@ -134,8 +134,14 @@
     my $preamble = $self->preamble;
     push @lines, $preamble->lines if $preamble;
 
-    push @lines, "--$boundary\n", $_->lines
-        foreach $self->parts('ACTIVE');
+    foreach $self->parts('ACTIVE') {
+	push @lines, "--$boundary\n", $_->lines;
+	if ($lines[-1] =~ /\n$/) {
+	    push @lines, "\n";
+	} else {
+	    $lines[-1] .= "\n";
+	}
+    }
 
     push @lines, "--$boundary--\n";
 
@@ -169,6 +175,7 @@
     {   foreach my $part ($self->parts('ACTIVE'))
         {   print $out "--$boundary\n";
             $part->print($out);
+	    print $out "\n";
         }
         print $out "--$boundary--\n";
     }
@@ -176,6 +183,7 @@
     {   foreach my $part ($self->parts('ACTIVE'))
         {   $out->print("--$boundary\n");
             $part->print($out);
+	    $out->print("\n");
         }
         $out->print("--$boundary--\n");
     }
Only in Mail-Box-2.060-5signed2/lib/Mail/Message/Body: Multipart.pm~
Only in Mail-Box-2.060-5signed2: Makefile
Only in Mail-Box-2.060-5signed2: out.msg
Only in Mail-Box-2.060-5signed2: pm_to_blib
Only in Mail-Box-2.060-5signed2/tests/20pparser: 34bodymp.t~
Only in Mail-Box-2.060-5signed2/tests: a
Only in Mail-Box-2.060-5signed2/tests/folders: Mail
Only in Mail-Box-2.060-5signed2/tests/folders: maildir.src
Only in Mail-Box-2.060-5signed2/tests/folders: mbox.win
Only in Mail-Box-2.060-5signed2/tests/folders: mh.src
Only in Mail-Box-2.060-5signed2/tests: msg-7152-1.txt
Only in Mail-Box-2.060-5signed2/tests: msg-8617-1.txt

--------------070903000501090303060605--