Re: Error (signal 11) during auto-train

I played around with the file a bit more.  The error only happens with 
Content-Transfer-Encoding: base64, and only when there are at least two 
linefeeds at the end, indicating there should be a body, but the body 
has no tokens.

Is this related to the bug posted earlier about skipping the 
tokenization of emails without a body, or is this a problem with the 
image tokenizer?  For the time being, what's my best workaround?


David Rosen wrote:
>
> I realized that I can reproduce the problem by running score on the 
> email message, so it's not an auto-train problem:
>
>    spamprobe -vv score t
>
> I simplified the file as much as possible and the following file 
> causes the error:
>
> -- begin --
> Content-Transfer-Encoding: base64^M
>
> ^M
> ^M
>
> -- end --
>
> This is the output:
>
> LOCKED FILE /home/drosen2/.spamprobe/lock
> WordData::setTodayDate: set date to 1284
> USING REQUESTED DATABASE TYPE '' PATH /home/drosen2/.spamprobe/sp_words
> OPENING DATABASE /home/drosen2/.spamprobe/sp_words
> OPENED DATABASE /home/drosen2/.spamprobe/sp_words
> DATABASE OPENED /home/drosen2/.spamprobe/sp_words
> CleanupManager added limit: count 2 age -1
> HDR: 'Content-Transfer-Encoding: base64'
> HDR: ''
> LINE: ''
> LINE: ''
> LINE: ''
> parseBody: begins
> DECODING BODY
> DECODED TO
> DECODED BODY
> PARSING PLAIN BODY TEXT LINES: 0
> caught signal 11: quitting
> Aborted
>
>
>
> David Rosen wrote:
>>
>> Hi,
>>
>> After installing SP 1.4a, I decided to rebuild my database, so that I 
>> could take full advantage of the new image tokenizing feature.  I am 
>> running auto-train on Maildir folders with this command:
>>
>>    spamprobe -c -v auto-train SPAM .X-Spam .X-Spam.Storage* GOOD 
>> .Business* .Current* .Friends* .Miscellaneous*
>>
>> After processing roughly 10% of the messages, I receive this error:
>>
>>    caught signal 11: quitting
>>
>> Searching about "signal 11", one cause in Linux appears to be memory 
>> full, and SP is using a large portion (8.5%), but not all, of the 
>> available memory of the machine.  I included a portion of the output 
>> at the end of this message.  Can someone shed some light on this?
>>
>> Thanks,
>> Dave
>>
>> P.S.  Been using spamprobe since 0.7 and it's still great!  Thanks 
>> Brian.
>>
>>
>> Here's a portion of the output:
>>
>> counting messages in file .X-Spam
>> file .X-Spam contains 198 messages.
>> counting messages in file .X-Spam.Storage
>> file .X-Spam.Storage contains 891 messages.
>> counting messages in file .X-Spam.Storage.2000
>> file .X-Spam.Storage.2000 contains 189664 messages.
>> counting messages in file .X-Spam.Storage.2001
>> file .X-Spam.Storage.2001 contains 0 messages.
>> IGNORING EMPTY FILE .X-Spam.Storage.2001
>> counting messages in file .X-Spam.Storage.2002
>> file .X-Spam.Storage.2002 contains 0 messages.
>> IGNORING EMPTY FILE .X-Spam.Storage.2002
>> counting messages in file .X-Spam.Storage.2003
>> file .X-Spam.Storage.2003 contains 0 messages.
>> IGNORING EMPTY FILE .X-Spam.Storage.2003
>> counting messages in file .X-Spam.Storage.2004
>> file .X-Spam.Storage.2004 contains 0 messages.
>> IGNORING EMPTY FILE .X-Spam.Storage.2004
>> counting messages in file .X-Spam.Storage.2005
>> file .X-Spam.Storage.2005 contains 0 messages.
>> IGNORING EMPTY FILE .X-Spam.Storage.2005
>> counting messages in file .X-Spam.Storage.2006
>> file .X-Spam.Storage.2006 contains 171 messages.
>> counting messages in file .Business
>> file .Business contains 0 messages.
>> IGNORING EMPTY FILE .Business
>> counting messages in file .Business.A
>> file .Business.A contains 431 messages.
>> counting messages in file .Business.A.A
>> file .Business.A.A contains 150 messages.
>> counting messages in file .Business.A.I
>> file .Business.A.I contains 1674 messages.
>> counting messages in file .Business.A.P
>> file .Business.A.P contains 196 messages.
>> counting messages in file .Business.A.S
>> file .Business.A.S contains 1932 messages.
>> counting messages in file .Business.A.W
>> file .Business.A.W contains 88 messages.
>> counting messages in file .Business.Jobs
>> file .Business.Jobs contains 65 messages.
>> counting messages in file .Business.P
>> file .Business.P contains 244 messages.
>> counting messages in file .Business.P.Other
>> file .Business.P.Other contains 78 messages.
>> counting messages in file .Business.S
>> file .Business.S contains 198 messages.
>> counting messages in file .Business.W
>> file .Business.W contains 42 messages.
>> counting messages in file .Current
>> file .Current contains 147 messages.
>> counting messages in file .Friends
>> file .Friends contains 3737 messages.
>> counting messages in file .Friends.Jokes
>> file .Friends.Jokes contains 34 messages.
>> counting messages in file .Miscellaneous
>> file .Miscellaneous contains 479 messages.
>> counting messages in file .Miscellaneous.Commerce
>> file .Miscellaneous.Commerce contains 246 messages.
>> counting messages in file .Miscellaneous.Finance
>> file .Miscellaneous.Finance contains 175 messages.
>> counting messages in file .Miscellaneous.Registrations
>> file .Miscellaneous.Registrations contains 168 messages.
>>
>> . . .
>>
>> COMMAND train-spam FILE .X-Spam.Storage.2000 DIGEST 
>> 8830fd20d01f2e28e9dd5ff670e46e0f SUBJECT F... PROCESSED
>> COMMAND train-spam FILE .X-Spam.Storage.2000 DIGEST 
>> e45c2f7e573c811b2478aa79b03692ea SUBJECT S... PROCESSED
>> COMMAND train-good FILE .Business.A.I DIGEST 
>> 28714820b2d5843b23609601cabafb90 SUBJECT Q... PROCESSED
>> COMMAND train-spam FILE .X-Spam.Storage.2000 DIGEST 
>> eca81a44226a97cd0d6912c9e9ab7344 SUBJECT w... PROCESSED
>> COMMAND train-spam FILE .X-Spam.Storage.2000 DIGEST 
>> d21e5e36fcbd7d69d72ab654c593bb6a SUBJECT T... PROCESSED
>> caught signal 11: quitting
>>
>>
>>
>



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642