Re: GerH source distro at hebbut.net: HOW TO compile/compare/merge for Bill et al

On Fri, Mar 28, 2008 at 01:01:06AM +0100, Paolo wrote:
> On Thu, Mar 27, 2008 at 10:54:04AM +0100, Ger Hobbelt wrote:
> > 
> > FYI: the latest src.7z is just out the oven. ;-)
> 
> 1st run [Piii, 450, 128, gcc-2.95.4, libc 2.2.5, Debian Woody 3.0]

2nd run, Athlon XP 2800, gcc (GCC) 3.3.5 (Debian 1:3.3.5-13), 
  ldd (GNU libc) 2.3.2, Debian 3.1 - aka Sarge
> 
> didn't work with  --enable-debug --with-regex=pcre:

nope, same noise.

> -x-x-x-
> 
>  --with-regex=tre:

hm - configure went to happy end but did not report libtre-dev was't 
installed. 'make' did though (of course). 
I'd expect the '#error "crm114 MUST be compiled with regex support. ...' 
during config, configure's conf_test.c should detect that and bail out right
there.

> one glitch:
> crmregex_tre.c: In function `crm_regversion':
> crmregex_tre.c:380: warning: passing arg 1 of `strlen' makes pointer from integer without a cast

didn't get this here

> 
> others?:

crm_errorhandlers.c:863: warning: unused variable `j'
crm_var_hash_table.c:227: warning: unused variable `value'
crm_var_hash_table.c:617: warning: unused variable `i'
crm_var_hash_table.c:1155: warning: unused variable `i'
crm_var_hash_table.c:1330: warning: unused variable `j'
crm_var_hash_table.c:1402: warning: unused variable `ic'
crm_var_hash_table.c:1425: warning: unused variable `ic'
crm_var_hash_table.c:1452: warning: unused variable `ic'
crm_var_hash_table.c:1487: warning: unused variable `ic'
crm_var_hash_table.c:1319: warning: unused variable `k'
crm_var_hash_table.c:1675: warning: unused variable `i'
crm_expr_match.c:31: warning: unused variable `k'
crm_osb_hyperspace.c:1136: warning: unused variable `textoffset'
crm_osb_hyperspace.c:1137: warning: unused variable `textmaxoffset'
crm_neural_net.c:489: warning: unused variable `i'
crm_exec_engine.c:333: warning: unused variable `i'
crm_expr_sks.c:1394: warning: unused variable `cmp'
crm_expr_svm.c:1481: warning: unused variable `cmp'
crm_test_hashes.c:743: warning: unused variable `feedlen'
crm_test_hashes.c:808: warning: unused variable `feedlen'
crm_test_hashes.c:864: warning: unused variable `feedlen'
crm_test_hashes.c:2981: warning: unused variable `i'
crm_test_hashes.c:2998: warning: unused variable `m'
crm_test_hashes.c:3051: warning: unused variable `m

> crm_test_hashes.c: In function `fake_hash':
> crm_test_hashes.c:2011: warning: decimal constant is so large that it is unsigned
> ... (lot of)

ya, same here

> 
> then src/crm_test_hashes seems to hangs at ~99% CPU at:
> 
> Testing: fake12_hash (82568478) (4)
> ...
> Mix success  42 bytes   8 initvals  required  18  trials
> Mix success  43 bytes   8 initvals  required  19  trials
> Mix success  44 bytes   8 initv
> 
> but after minutes it finishes. Wew :P.

same (but partial printout was/is due to buffering)
for curiosity, took real    1m42.530s
didn't check crm_test_hashes' details, so didn't check its trace either (for
now)

> 
> Then megatest ... did not succeed [removed some bla-bla]:

... which took 'real    0m33.661s' here:
 
> ...
> @@ -689,6 +697,0 @@
> - type Q 
> -CLASSIFY fails; success probability: 0.0000  pR: -10.6507
> -Best match to file #1 (q_test.css) prob: 1.0000  pR: 10.6507  
> -Total features in input file: 176
> -#0 (i_test.css): features: 140946, hits: 693, prob: 2.24e-11, pR: -10.65 
> -#1 (q_test.css): features: 133586, hits: 5022, prob: 1.00e+00, pR:  10.65 
> @@ -695,0 +699,65 @@
> + ****  Default (SBPH Markovian) classifier 
> +./megatest.sh: QUICKREF_mt_ng_reference_1.input: No such file or directory
> +./megatest.sh: INTRO_mt_ng_reference_2.input: No such file or directory
...

yep same - megatest target is broken. Actually, such QUICKREF and INTRO don't
exist, they should be the usual .txt; or you've left them out of the 7zball.
Or I missed something.

Anyway, temp fix:
$ sed -e 's,_mt_ng_reference_[12]\.input,.txt,g' src/*.crm src/megatest.sh -i

Now with .txt, test takes 16m44s and seems to work, except for diffs in 
features/hit counts (due to modified .txt,right?), and for NN that bombs out:

+**** Neural Network Classifier 
+ neural: failed to converge within the training limit.   Beware your results.
+This happened at line 2 of file (from command line)
+(runtime system location: crm_neural_net.c(2169) in routine: crm_neural_net_learn)
+The line was:
+--> learn < neural refute fromstart > (q_test.css) /[[:graph:]]+/.
...
@@ -982,2 +1052,2 @@
-CLASSIFY fails; success probability: 0.241661  pR: -62.5048
-Best match to file #1 (q_test.css) prob: 0.7583  pR: 62.5048  
+CLASSIFY fails; success probability: 0.500000  pR: 0.0000
+Best match to file #0 (i_test.css) prob: 0.5000  pR: 0.0000  
@@ -985,2 +1055,2 @@
-#0 (i_test.css): prob: 2.42e-01, pR: -62.50
-#1 (q_test.css): prob: 7.58e-01, pR:  62.50
+#0 (i_test.css): prob: 5.00e-01, pR:   0.00
+#1 (q_test.css): prob: 5.00e-01, pR:   0.00

and

+ neural: failed to converge within the training limit.   Beware your results.
+This happened at line 46 of file alternating_example_neural.crm
+(runtime system location: crm_neural_net.c(2169) in routine: crm_neural_net_learn)
+The line was:
+--> learn [:_nl:] <neural fromstart> (i_test.css).     
... 
+./megatest.sh: line 494: 12593 Segmentation fault      ./crm114 alternating_example_neural.crm


Clump / Pmulc fails as well:

+**** Clump / Pmulc Test 
+
+crm114: *UNTRAPPABLE ERROR*
+
+Better start screaming, guv', since the software's just gone critical:
+assertion 'n >= 2' failed!
+Sorry, but this program is very sick and probably should be killed off.
+This happened at line 3 of file (from command line)
+(runtime system location: crm_expr_clump.c(816) in routine: thresholding_average_cluster)
+The line was:
+--> clump <bychunk> [:one_paragraph:] (i_test.css) /[[:graph:]]+/. 
...
+crm114: *UNTRAPPABLE ERROR*
+
+Better start screaming, guv', since the software's just gone critical:
+assertion '!(scale <= 1.19209290e-7F && scale >= -1.19209290e-7F)' failed!
+Sorry, but this program is very sick and probably should be killed off.
+This happened at line 3 of file (from command line)
+(runtime system location: crm_expr_clump.c(840) in routine: thresholding_average_cluster)
+The line was:
+--> clump [:one_paragraph:] <bychunk> (i_test.css) /[[:graph:]]+/.


well, no big surprise, those are 'experimental' right? better leave 'em 
out of 'production' though.


[meanwhile, corrected megatest still running on my Piii/450 ... let's see
whether it'll ever end before burning the CPU :P ]

-- 
paolo

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace