RE: Database for greylisting plugin.

Bruce Guenter wrote:
> One con to using the Berkeley/SleepyCat db libraries is that they
> change their file format every other release, and you can't even read
> the old databases with new libraries.  On the other hand, they work,
> and they work well.

Well we won't go there then. <grin>

> If you can arrange to have either completely empty files or small
> numbers of files with larger contents, then I wouldn't worry about the
> caching hit.  Either using the filesystem as the database or adding a
> database layer, the OS is going to have to cache the data, and the
> file names themselves are not any more expensive than putting it in a
> database.  The OS will want to cache an inode to provide timestamp
> information, which starts to get a bit bigger, but still not a huge
> issue.
> 
> Using the filesystem is vastly simpler for debugging and emergency
> maintenance, as standard file tools work.

This makes a lot of sense.

If it was designed to do basic greylisting (sender-receiver-ip) using the
filesytem as a database (emtpy files and use mtime), then it cannot
differentiate between a triplet combination that was greylisted 8 hours ago
(and never returned) and one that has already returned (which should still
be whitelisted). This is, of course, unless the whitelist was separate from
the greylist and whitelist was checked first. I am not sure I want to do
that. I think I would prefer all entries in one database "table" so the I/O
is kept down to the minimum.

Of course, I might just be totally ignorant of how to use the filesystem as
a database with empty files.

Does any think that the queue algorithm in qmail would be of any use? Or how
about tai coding expiry time into filename? Shudder... would mean a grep to
find a match of triplet perhaps.

It seems from my reading that a smarter system is possible with a record
oriented database. Then separate fields can record first access, last
access, hit/block/fail counts, expiry times, and whitelist flags. This extra
info would allow things like extended expiry times for ips that meet certain
criteria (sort of automatic whitelisting), and selection of a grouping
function to handle server farms not sending from the same ip, and delaying
of 451 error to cope with brain dead sender verification scheme. These to
name a few. With one I/O.

Does anyone know of any better schemes for using the filesystem that might
be useful? Is there more info than the mtime of a file that could be used
and manipulated?

Thanks
Graham