Re: Using crm114 for non-email discrimination

On Thu, 26 Feb 2009 20:14:46 +0000 (UTC)
Lloyd Zusman <[email protected]> wrote:

> I'm a satisfied user of crm114 for email spam filtering. Now, I'd like
> to utilize this excellent piece of software for another purpose: to
> help remove spam from an online forum I'm managing. I've looked around
> for examples of non-email uses of crm114, but the only one I found
> (the CV discriminator mentioned in the "coolthings" area of the main
> crm114 site) is a dead link.
> 
> I'm wondering if any of you could point me to some docs or discussions
> for how to configure crm114 for non-email use.
> 
> The filtering I want to perform will only be on the bodies of the
> messages. I want to completely ignore the titles and the sender.
> Messages will be "flagged" as being unacceptable, and when this is
> done, I want to "learn" them as spam.
> 
> A simple "cookbook" for how to do this for non-email would be ideal.
> Short of that, any docs or discussions would be quite helpful.

I don't think that there's any existing documentation on this, but I'll
give you a big hint on one way to do this...

An HTTP POST request is an RFC 822 message.

Most of the time your forums are going to POST an URL-encoded
submission to a(nother) page on the site. You'll have to rename the
destination page and create a CRM script by the original name of the
destination to pass the (filtered) post data to the final destination.

Your CRM script will receive the post data on stdin, run your
classifier with "+" as the tokenizer, and submit those POSTs that pass
to the final destination.

The trick is to hide the final destination from Spammers. The best way,
if possible for your setup, is to syscall the destination script like
this:

	syscall (:*:_dw) (:_dw:) /php submit.php/
	accept

This replaces the (filtered and approved) contents of the POST request
with the HTTP response that you would have generated if you hadn't
interposed your filter then places it on stout for the web server to,
well, serve.

See the "CGI Scripting with CRM" thread begun Feb 6. You don't need to
parse the URL encoding if you are going to submit it to a CGI-like
program, but the sample script will help you understand what is going
on in your environment.

Chris

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H

_______________________________________________
Crm114-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crm114-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iQEcBAEBAgAGBQJJpv9XAAoJEASgqNsqZfCHQX8H/04+aZiekJaIYk4EVgjlFvTe
SuVu0nU+ZtRnqNtTHF7lQdNvOSZ1EUYvU/7P4sETYOeNUuj0DW4Zf5exTiN2G6ki
iU7jxLl/kHGU3aKW/ZKvkbK/mb/9qMjumLUdap8kpo9gFdGiAXKiosb+HEdcic8k
xLY9o72UYG4OzJOmf6QF5ToZkfdzFjOLKEC10X27dXMPNhgZ+W60YsBtJi3msqK4
gluC4GQIgcO6+wlMGwP57aZXLhJZrL0KFehV3kLvV4faHAxKSRt2ZSgmOoCtHGsq
SZuuApbDF/8AZto2E8KFvSOmrxF0BFO6vdv47XXGZ2taZffxy9cf5qNGjIIfc1E=
=4NsC
-----END PGP SIGNATURE-----