RE: KiokuDB::Backend::BDB and replication

From: Yuval Kogman [mailto:[email protected]]
=20
> Hi,
>=20
>=20
> 2009/5/24 atomicstack <[email protected]>:
> > I notice BerkeleyDB::Manager has attributes for replication, but no
> mention
> > in the documentation. Has anybody had luck using it in a replicated
> > environment? I'm tinkering with MemcacheDB but the tests seem to =
fail
> when
> > BDB::Manager calls BerkeleyDB::Hash->new. My XS knowledge isn't =
great
> so I'm
> > not sure where/how to start digging into BerkeleyDB to see if it's
> honouring
> > the replication flags properly...
>=20
> That flag is probably a leftover from my experiments, it's not
> supported yet.

And it isn't supported as yet in my BerkeleyDB module.

> Adding replication to the BerkeleyDB module is possible, but requires
> quite a bit of XS work, unfortunately.
>=20
> There are several steps required to support replication in BerkeleyDB
> (the module):
>=20
> 1. Berkeley DB (the library) support 4 modes of memory management.
> BerkeleyDB.xs uses only one of these,=20

I assume you are referring to ENV->set_alloc and DB_DBT_MALLOC, =
DB_DBT_REALLOC,... ?

> where the library allocates
> everything (instead of using a callback for instance). Unfortunately
> this mode is incompatible with the threaded environment,=20

My BerkeleyDB module is *NOT* thread safe at the moment, especially =
where transactions are concerned. Major surgery to BerkeleyDB.xs will be =
required to make it work.

> which is
> necessary for replication (bdb spawns a background thread for this).
>
> 2. The XS api for replication is not written yet. This is
> comparatively easy though.
>=20
> 3. the replication API doesn't support automatic forwarding to the
> current master, only finding out who the master is. This means that
> the RPC layer also has to be supported. To allow transparent writes on
> the slaves all writes must be forwarded to the current master and then
> replicated to the slave. The RPC api is also not too hard to expose.
>=20
> 4. After RPC is available, writing a layer to automatically forward
> all write operations to the master is still a big undertaking. For
> instance if there is an open transaction with write operations that
> probably means all reads need to be done off the master too, so to
> benefit from replication this might not be a good approach, and
> something a little more explicit is necessary.

When I get round to adding replication, I intend to start with the =
replication manager interface only. Whether I get to any of the lower =
level replication api remains to be seen.

Paul