Re: Re: Error (signal 11) during auto-train

David Rosen wrote:
> I played around with the file a bit more.  The error only happens with 
> Content-Transfer-Encoding: base64, and only when there are at least two 
> linefeeds at the end, indicating there should be a body, but the body 
> has no tokens.

Thanks for all the digging you did on this David!  I was able to 
reproduce and eliminate the problem very quickly using your sample.


> Is this related to the bug posted earlier about skipping the 
> tokenization of emails without a body, or is this a problem with the 
> image tokenizer?  For the time being, what's my best workaround?

In terms of the location of the problem in the code - no.  Certainly 
there are similarities in concept since they both involved empty bodies. 
  However the no tokenization bug was just that I skipped a message 
completely if it had an empty body rather than processing its headers 
first.  A logic error.

The crash was related to a boundary condition that I hadn't caught in 
any of my tests - a malformed MIME message with no body.  In that case 
the parser didn't properly recognize the end of the message and tried to 
parse beyond it.  Both were easy to fix once identified.  Of course I 
had assert()s in the code that detected this problem the first time I 
ran against the sample - but asserts are off in production builds for 
performance reasons...

I don't think the crash has any security implications since it only 
happens if the message body contains no data.  Hard to inject any code 
if you can't provide any data.

All the best,
++Brian




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642