Re: email header wrapping and truncation

Hi Peter,

On 2017.07.21 17:09, Peter Bloomfield wrote:
> Hi Jack,
> 
> On 07/21/2017 03:18:56 PM Fri, Jack wrote:
>> Greetings all,
>> 
>> This is a problem that has bitten me in the past, but I finally  
>> tracked it down, with sufficient examples to demonstrate the problem.
>> 
>> In most cases, email headers (in the actual message) are
>> 
>> Header:  content
>> 
>> However, for some long values, it appears as
>> 
>> Header:
>>  content
>> 
>> If the content is long enough, you get
>> 
>> Header:
>>  first part of header, but tr
>>          uncated and wrapped.
>> 
>> It seems that in this last case, Balsa only sees the first line as  
>> the value of that header ("first part of header, but tr") which can  
>> cause a problem, if, for example, it is the "Reply-to:" header.  I  
>> now have two examples of this actually happening.  One is marketing  
>> messages I used to get from consumer cellular.  The other is from  
>> the gentoo.org mailing list system.  If I send a subscribe reqeust,  
>> it replies with a message I need to reply to to confirm the  
>> request.  However, it's Reply-to is of the third form above, and not  
>> noticing the truncation, my first reply just got me an error bounce.
>> 
>> Most interesting however, is that when I went to find the comsumer  
>> cellular example, I found a fourth type:
>> 
>> Header: Initial part o
>>  f header, with more here, and th
>>          e remainder here
>> 
>> In this case, Balsa gets the entire content of the header without  
>> problem.
>> 
>> I suppose the difference between the third and fourth types depends  
>> on one of the mail handlers between the sender and the one I get the  
>> message from by POP3.  However, I don't know if consumer cellular  
>> changed their system, or something changed someplace else in  
>> between, or perhaps some change in Balse (or a library) as all  
>> examples I can find of the third type are older than the fourth type.
>> 
>> Sorry for rambling, but hopefully this explains my issue enough for  
>> someone to either confirm it or point out what I'm doing wrong.
>> 
>> Let me know if you need any other info, such as versions of  
>> libraries my Balsa is compiled against.
>> 
>> Thanks for any help.
>> 
>> Jack
> 
> Sorry to read about the problems you're seeing!
> 
> Those continuation lines (beginning with WSP: space or tab) are all  
> supposed to be 'unfolded' by concatenating them with the 'Header:'  
> line. Balsa hands off the whole job of parsing the message to GMime,  
> which then provides unfolded headers on request. The GMime code that  
> handles unfolding has been stable for many years.
> 
> Perhaps the lines that don't get concatenated have some other  
> non-printing character before the WSP. Where do you see the folded  
> headers? In an mbox? Can you check for other spurious characters?
> 
> Best,
> 
> Peter
My inbox is an mbox, but I generally move almost everything to maildir  
folders, so that's where all my recent examples come from.  I haven't  
noticed any odd characters (just looking with grep or in emacs) so I'll  
have to dig back in and try some "od -c".  Since the only thing exposed  
would be email addresses, I suppose I can extract different examples  
into an attachment.

I've had gmime 2.6.23 installed since February, but had this problem  
with Consumer Cellular early 2016 and with Gentoo mailing lists just  
the past few days.  I suppose "Reply-to" is one of the few headers  
(other than "From" and "CC" which frequently get used, but only  
infrequently gets a really long value.  In the case of the mailing  
lists, it's not a "real" address, but is parsed by the list software so  
all it needs is that address and the subject to know what to do.

Jack