Hang during learning

"Angus McIntyre" <[email protected]> Wed, 9 Jun 2010 15:15:43 -0400 (EDT)
Newsgroups gmane.mail.spam.crm114
Message-ID <[email protected]>
I have a very simple learning script, which consists of:

   {
   learn <osb unique microgroom> (:*:_arg2:)  
   }

When I invoke it with:

   cat samplefile | crm learn.crm cssfile

(file and directory names have been changed to protect the innocent), it
periodically hangs.

The files being learned are plaintext (they're actually spam emails), and
the stall occurs with a css file that has been trained on several tens of
thousands of example files.

The file being read when the stall occurs varies, so I don't believe the
problem is caused by something specific to the file contents, path or
permissions.

The css file should have ample room on the filesystem to expand if needed.
The host on which the script is running has a total of 3GB of physical
memory.

Deleting the css file and starting over allows processing of the directory
to complete without problems. However, as I train the css file on more and
more examples, I will eventually reach a point where the learning script
hangs again. As far as I have seen the learning process never terminates.

This is with 20100106-BlameMichelson (TRE 0.8.0 (BSD), but also occurred
with earlier versions.

Has anyone else encountered similar problems, and are there any possible
workarounds?

Thanks, Angus


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo