Re: libcrm114 ready - but can't get into sourceforge

On 07/05/2011 07:45 PM, [email protected] wrote:

> So, until that gets figured out, I will mail it out to anybody who
> asks.  Who wants it?  Send me email!  Be advised that it's not
> a tiny .gz file.  :)

so, can I have it please.

With my other questions I was not clear enough, will try to clarify:

> Easier than that.  IF (and big IF) you are going to restore the
> data back onto a machine with the same architecture (i.e. the same
> host, running the same OS and same libC layout of structures) then
> a binary read/write works fine.
> 
> In fact, the recommended procedure is to write (or read)
> p_db->datablock_size blocks of data to the file system of your
> choice. 

Yes, I am talking about the situation where I save the p_db's on the
same machine for later reuse. So I am interested in binary writing and
reading. It is obvious how to write the data from p_db to a file - take
the pointer, look at p_db->datablock_size and write that many bytes.
Now what shall I do when I want to reuse this saved data? Call stat() to
determine the number of bytes in the file, malloc that much memory, and
read the data from the file to that memory, period? No extra
cleanup/initialization necessary?

About mmap(): I was thinking that *if* I am not planning to "learn" or
otherwise modify the p_db, I can mmap() the previously saved file and
use this memory as the p_db (instead of malloc+read). Correct? It should
be a little bit easier and (negligibly) more efficient than malloc+read.

Now, why I was talking about re-reading the saved p_db's?
My plan is this: I want to have one process possibly with multiple
threads that will classify *many* documents in its lifetime. Think
milter process that scans all incoming mail. It will never "learn" so it
hopefully can mmap() the previously written binary copies of p_db's. I
also expect to have a learning procedure, like "this message was
misclassified, it is not spam". On my small system, that will be done
only a few times a day, so I plan to have it "one process invocation =
*one* learn operation". Every time I fire this re-learning operation I
will get a new state of the p_db's. And then I write them out. I suspect
that I will need to write them to the new copies of the files, and tell
the milter process that it needs to re-read (or re-mmap) the new files
(or simply restart the milter). [Ideally, I would prefer to lock() the
mmap()ed files, update them by the learning process, release the lock,
and be sure that when the classifier process obtains the lock for the
next classification round, it will "see" the new data and happily use
it. But I somehow suspect that this will not be possible.]

Am I making sense now?

Thanks,

Eugene

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2

_______________________________________________
Crm114-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crm114-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4TPgoACgkQfrw/cIw6UWm53gCfVemU+uQoqVmhVAAANeBOXwoa
/doAn2dDNlsvGqhczBAB5lm5CdaDvimC
=+vew
-----END PGP SIGNATURE-----