Re: problem with trainin since upgrade to blamethesegfault

>  This one:
>  http://crm114.sourceforge.net/tarballs/crm114-20070810-BlameTheSegfault.src.tar.gz
>
>  with the exception that I patched it slightly to give a bit more error
>  reporting as per yours and Paolo's suggestions.

okay. sounds good.

So far, nothing popping up on the brainscan here. :-S


Hence, it's back to good ol' fashioned printf debugging here, I fear. Oh dear...


Let's walk down the list:

File:
mailfilter.cf

can you provide a copy of the line matching
  :text_cache:
please?

Should read something like:

  :text_cache: /reaver_cache/

which is a relative path to reaver_cache.


Attempt 1:

Before we go on, when you convert this to a 'full path', i.e. something like:

  :text_cache: /\/var\/mail\/reaver_cache/

(note the escaped slashes, which are actually wrong, because this is
NOT a CRM script but a binary data file (see other email of mine in
crm114-dev), but bash will save our day anyway, so go ahead.)

Now if you run the training, what's it say?

If error gone, tis is a definite 'HMMMM', if not, I'm VERY interested
in the path reported at that ifn= line in crm itself.

Anyhow, no matter this result, please indulge me and do Attempt 2 too, thanks!


Then we go on:

Attempt 2:

restore or keep that full path up there, no matter. You may decide to
revert back to what it was before, just to keep changes to a minimum.


No its editing time, and we're doing it bluntly.

Before we go, decide where you want the new logfile to go; the one
we're going to write during this ordeal. Let's assume we're going to
put it in the current directory and call it 'jason_debugging.log'.

Next, better backup mailreaver/maillib/mailtrainer.crm.

Then open up mailreaver.crm and hunt down every line in there where
you find this

  :*:text_cache:

and duplicate that line, replacing the command plus everything up to
and including the first '/' for the first of each pair into

  output <append> [jason_debugging.log] /LOG @line X :

so the first one would become, for example:

        output <append> [jason_debugging.log] /LOG @line 384 :
:*:text_cache:\/texts\/:*:cacheid:/
        isolate (:long_cacheid:) /:*:text_cache:\/texts\/:*:cacheid:/

and a little further down a far more important one:

  output <append> [jason_debugging.log] /LOG @line 542 :
:*:cache_dupe_command: :*:text_cache:\/texts\/:*:cacheid:
:*:text_cache:\/known_good\/:*:cacheid: /
  syscall /:*:cache_dupe_command: :*:text_cache:\/texts\/:*:cacheid:
:*:text_cache:\/known_good\/:*:cacheid: /

and another important one immeditately following this one:

  output <append> [jason_debugging.log] /LOG @line 547 :
:*:fileprefix::*:trainer_invoke_command: --fileprefix=:*:fileprefix:
--good=:*:text_cache:\/known_good\/:*:cacheid:
--spam=:*:text_cache:\/empty\/ / () (:mailtrainer_output:)
  syscall /:*:fileprefix::*:trainer_invoke_command:
--fileprefix=:*:fileprefix:
--good=:*:text_cache:\/known_good\/:*:cacheid:
--spam=:*:text_cache:\/empty\/ / () (:mailtrainer_output:):

Note that I changed the line number to match the actual line number of
the source file so we get a logfile which will tell us exactly what is
done where in that crm script.

There's more of them, please do them all without consideration. Brute
force it is.


BTW: There's also some use of that variable in maillib, but I don't
think we'll have to go all that far.


Then open up mailtrainer.crm and find these two lines near the top ~ line 220:

  isolate <default> (:spam:)  /ERROR!!!/
  isolate <default> (:good:)  /ERROR!!!/

and ADD AFTER those these lines:

  output <append> [jason_debugging.log] /TRAIN @line 225 : training
good dir: :*:good:/
  output <append> [jason_debugging.log] /TRAIN @line 226 : training
spam dir: :*:spam:/

Then it's on to finding these two:

  match [:good:] (:gooddir:) /^.*\//
  match [:spam:] (:spamdir:) /^.*\//

and adding these following those:

and ADD AFTER those these lines:

  output <append> [jason_debugging.log] /TRAIN @line 268 : STRIPPED
training good dir: :*:gooddir:/
  output <append> [jason_debugging.log] /TRAIN @line 269 : STRIPPED
training spam dir: :*:spamdir:/



We're getting close, but then we have the whole darn thing 'instrumented'...

Look further through mailtrainer.crm for :good: and :gooddir: usage:
look for this line:

        syscall /ls :*:good: / () (:goodfiles:)

and duplicate that to prepend it with another of those output // log
lines like we've done before, like this (remember to start each
logline with TRAIN now, so we can see in the logfile which crm source
did what exactly)

        output <append> [jason_debugging.log] /TRAIN @line 341 : ls
:*:good: / () (:goodfiles:)
        syscall /ls :*:good: / /[Windows-MS]dir \/b :*:good: / () (:goodfiles:)

Now something EXTRA: after that 'ls' syscall there, add another
logline like this:

        output <append> [jason_debugging.log] /TRAIN @line 341 : ls
output =\n:*:goodfiles:/

so we can have look at what 'ls' brought us (now that I look through
mailtrainer, I have a hunch...).

We're close to done, just a few more. mailtrainer doesn't lend itself
for the brute force approach of logging a var everywhere all that
well, so this takes a little tinkering.

First, there's :clip_filename: -->

find this line in the mailtrainer source:

            call /:clip_filename:/ [:gtl:] (:filename:)

(you should find it at three spots!)

and duplicate each line, replacing the SECOND line of each pair with
an output statement so this stuff looks like this when done:

            call /:clip_filename:/ [:gtl:] (:filename:)
            output <append> [jason_debugging.log] /TRAIN @line 341 :
call clip_filename(':*:gtl:') --> ':*:filename:'/

WARNING: note that I editing the logline with :*: constructs so that
we can see what got in (:*:gtl:) and what came out (:*:filename:)

Then there's one last thing to do: a blunt duplicate+log action to end it all:

Find any of these lines, matching

  input [

and where there's any mention of

  :filename:

in there. Duplicate each, and replace the FIRST of each pair with a
logline, just like this:

        output <append> [jason_debugging.log] /TRAIN @line 1082 :
input [:*:gooddir::*:filename: 0 :*:decision_length:]/
        input [:*:gooddir::*:filename: 0 :*:decision_length:]

Note that I kept the [] brackets in there; should be harmless, as
we're inside a /.../ argument here.


Now we're done.

I hope everything is okay, save and run and see what the logfile brings us.


My hunch? It just _might_ be that one of those lovely

                trap /unable to read-open/

lines do not kick in.

Plus I'll put another chip on square 42, where it says that there
might be something about some code in there looking like this:

                input [:*:gooddir::*:filename: 0 :*:decision_length:]
                trap /unable to read-open/
                input [:*:filename: 0 :*:decision_length:]
                trap /unable to read-open/
                output /\n COULDN'T READ THE GOOD FILE ':*:filename:'\n/

while in other spots it's only this:

                input [:*:filename: 0 :*:decision_length:]
                trap /unable to read-open/
                output /\n COULDN'T READ THE GOOD FILE ':*:filename:' \n/

Thirdly, all the remaining black chips are going to be stacked on #13,
where it says there's something fishy inside Mr. Clip_Filename, at
least when combined with the 'ls' output as provided earlier inside
mailtrainer.crm.

So far, so good. Now give me some good odds here. ;-)

I placed my bets. If everybody's done & ready?  'Rien ne vas plus!'
... and please spin the cronjob/crm wheel if you would, maestro! ;-))
Fait vous jeux, mes amis!



(I don't play poker or other regular betting games; instead I
sometimes wager a little something on the software debugging business
- it's _the_ way to play high-stakes Russian Roulette in a country
where you can't carry a .45 around without several permits plus a
government license to kill or a glorious criminal record. If we'd have
bookies that would cater to this sort of fetish, I'd probably make 'em
filthy rich. ;-) )





... and the log output issssss?


(The thing we're looking for - I'm betting heavily on this - is a
'jump' from files and directories being reported as
'reaver_cache/known_good' to suddenly
reaver_cache/known_good/reaver_cache/known_good'. The 'hunch' is that
'ls' may produce something that didn't happen before or is otherwise
'unexpected', resulting in clip_filename to choke.
If all that isn't what going on, there's a VERY big chance the traps
in there don't catch what they should as Will Bill And His Mates
didn't expect crm114 to throw a certain 'new' nonfatalerror. If THAT
is the case, it's 'hah!' because that's what you get when you don't
litter your code with errorchecks where they belong up front, but
allow error checking to 'grow organically' on you. But I expect Vicar
Ger (who doesn't eat his own sermons) to remain in the closet for this
one; it's probably the ls/clip going belly up for some odd and yet
unknown reason.)

Either way, like Anton Ego said:

  SURPRISE ME!   <waves & smiles />






-- 
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web: http://www.hobbelt.com/
 http://www.hebbut.net/
mail: [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace