Re: A memory leak and the reading of uninitialized bytes in 3.10.1... patch attached

On 02.09.2011 08:14, Ladar Levison wrote:
> One 8/28/2011 6:12 AM, Stevan Bajic wrote:
>> Should be fixed in GIT
> Further testing found another code path that leads to a memory leak. I 
> didn't realize these lines were inside a for loop the first time 
> around; and without the loop a free call wouldn't be needed.
>
Fixed in GIT. Thanks for the patch.


>> Most people use the Apache SpamAssassin corpi for testing or the TREC 
>> corpi.
> I was wondering if there was an actively maintained corpus available. 
Does it mater if the corpus is actively maintained or not? For testing 
email classification the only thing that count is the quality of the corpus.


> Something like compressionratings.com -- but for email classification. 
> Something I could test my configuration/build against to see where it 
> ranks.
>
You can use dspam_train for that. It will split out the score after it 
has finished with training. Those numbers can be used to compare against 
others that have used the same corpus for training.


>> You could use dspam_train and use dspam_stats to set/reset the snapshot.
>
> Yeah! But I'll need a script or tool to automate the testing process? 
> I use the library API so I'm not a good person to write such a test 
> script.
>
dspam_train does this (see man dspam_train):
NAME
        dspam_train - train a corpus of mail


SYNOPSIS
        dspam_train [username] [--client] [-i index|spam_corpus 
nonspam_corpus]


DESCRIPTION
        dspam_train  is used to train and test a corpus of mail (in 
maildir or MBOX format). This tool will present each message to DSPAM 
for a classification and then retrain only if the
        message was incorrect. This provides close to real-world 
training and should be used to build pretrained databases. Upon 
execution, the tool will automatically determine the ratio
        of  spam:nonspam and train based on that ratio to ensure both 
corpora are trained consecutively. This tool can also be used as a test 
jig to measure the efficiency and accuracy of
        a particular corpus against DSPAM in a given configuration.


OPTIONS
        --client
               If specified, DSPAM is used in client-server mode.


        username
               Specifies the user to train, if omitted the current user 
name is used.


        -i index
               Use a index file instead of the usual spam_corpus and 
nonspam_corpus.

               index :  Path to the index file having the following 
format per line:
               [class] [path to message]


        spam_corpus
               Specifies either the pathname to the directory containing 
the corpus of spam, with each in a separate file (e.g. maildir format) 
or a path to the mailbox in the tra-
               ditional Unix MBOX format.


        nonspam_corpus
               Specifies either the pathname to the directory containing 
the corpus of nonspam with each message in a separate file or a path to 
the mailbox in the traditional Unix
               MBOX format.


EXIT VALUE
        0      Operation was successful.
        other  Operation resulted in an error.


COPYRIGHT
        Copyright (C) 2002-2011 DSPAM Project
        All rights reserved.

        For more information, see http://dspam.sourceforge.net.



>>
>> I don't understand what you mean with this? Are you trying to get a
>> certain score/result that you can compare with the other DSPAM
>> users/developers?
>
> Exactly! If I run my build against an identical corpus I should get 
> identical results!
Well... yes and no. You can change a lot of stuff inside dspam.conf that 
influences the results.


> If the results vary, I know its time look for bugs. The goal is to 
> catch a string function or memory allocator that behaves differently. 
> I can always decide the deviation is small enough to ignore... but 
> only if I have results for comparison.
>
String functions or memory allocator errors? In what sense? Do you think 
that your build would trigger such an error while it will not trigger on 
another setup?


>> I don't know how other benchmark their setup (and if they even do
>> benchmark their setup)? I myself have developed over the years my own
>> testing and training method. I don't use stock DSPAM methods at all. I
>> guess other DSPAM users/admins have established their own test and
>> training procedures as well.
>
> I was hoping to find the tools/scripts/notes to test my 
> build/implementation. It would be nice if those tools became part of 
> 'make check'. If the testing bits take up too much space, just 
> distribute them as a separate tarball. The libxml2 project uses that 
> strategy. Each release tarball is paired with its own test tarball. 
> Check out ftp://xmlsoft.org/libxml2/ for the files.
>
As I have written in the past: It is hard to do the same with DSPAM 
because of the storage backend that DSPAM uses. You can not just make 
check during configure. It is not that easy.


>> This is difficult since the backend is configurable with ./configure but
>> it is most likely not initialized and a 'make check' would require to
>> have a properly configured backend (with all the schema and access
>> already setup), which is not available on a fresh/new setup during
>> compile time.
>
> A good start might be to compile the command line utilities and/or 
> test programs using with the file system storage driver.
Not everyone is compiling that storage driver. Some one might choose to 
just use MySQL and nothing else.


> If the check variants are compiled in response to 'make check' and 
> stored inside the test folder they shouldn't cause any problems. Then 
> its just a matter of automating the test process. And if the results 
> are stored under the build tree they could be purged easily pruged 
> with 'make clean'. ClamAV ships with a test corpus and 'make check' 
> will test the corpus against the command line tools. It checks whether 
> a reasonable amount of memory was needed; that the program finished 
> quickly and most importantly that it generates the expected 
> classification.
>
None of those tools have artificial intelligence like DSPAM has. ClamAV 
is just checking hashes. If DSPAM would work the same way then making a 
check suite would be easy, but it is not. We could however go on and 
code a suite that checks if the test "I am a test" is producing the 
proper result when using the various tokenizers. But that's all.

> This strategy could be used to test libdspam and could allow limited 
> testing of the command line utilities. IMO thats the most important 
> chunk of code.
>
> When time allows; adding logic to test different storage 
> configurations shouldn't be possible. Just write the check script with 
> the assumption a valid test database available. If the dspam user 
> won't connect to the localhost using the password 'bajic' then 'make 
> check' simply fails.
>
'bajic'? LOL. I am just a coding monkey. That's all. I for sure don't 
want to taint users database backends with my family name as password.
If you are so ultra giga horny for tests then I could build a database 
for MySQL and one for PostgreSQL that has tokens and dump that data and 
upload it to Sourceforge. Users could then use that data to do some 
limited testing.


> If you wanted to get a little more complicated, try executing the 
> RDBMS binary against a localized config file. Then initialize your 
> blank database schema and listen for connections via a file socket or 
> named pipe. Since the database files are stored inside the build tree, 
> they can be pruged and recreated each time 'make check 'is called. 
> Checkout the MySQL tarball and run "./configure; make && make check" 
> for the details.
>
> P.S. If anyone else decides to test DSPAM using Valgrind, the current 
> release (3.6.1) will complain about glibc str functions reading dirty 
> memory via aligned reads. The issue is fixed in the valgrind code 
> repository -- for those willing/able to compile a 3.7.0 snapshot.
>
> _______________________________________________
> Dspam-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/dspam-devel

------------------------------------------------------------------------------
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev

_______________________________________________
Dspam-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspam-devel