Re: [bdbxml] Java dbxml: Differences between locks, lockers, lock objects? how do you reclaim them?

Adrien,

First, the 30-second tutorial on locks...

In Berkeley DB terminlogy, a "locker" is something like a database, a
transaction or a cursor.  Berkeley DB "locks" are owned by a "locker"  
and
generally lock pages of a database.  There are other kinds of locks.

Translating to BDB XML, lockers are associated with Containers, which
own database handles, as well as documents, which may own cursors.

Lockers and locks only exist if you are creating and using a Berkeley
DB environment, and they exist even if your application does not use
transactions, in order to support some level of concurrent access.

Important locks/lockers in BDB XML:

In BDB XML 2.1.8, without transactions, there is a "locker"  
associated with each
active XmlDocument (which can be referenced by XmlValue
and therefore XmlResults objects).

Each locker may have a number of pages "locked"
that represent all or part of a given document.

Locks are not released until the locker disappears, which means
deleting the XmlDocument object.

However, if transactions are in use, the transaction
itself is the "locker" and there are no lockers associated with
documents, and locks are all released when the transaction
is committed or aborted.

Java...

Java is a special case, because there is no built-in destruction,
and resources other than locks/lockers are held by an XmlDocument
(memory).   This is why it's necessary to explicitly delete()
XmlDocument, XmlValue, and XmlResults objects in Java
in order to reclaim locks, lockers, and memory.  The same
is true for some of the scripting language interfaces.

End of 30-second tutorial...

> Hello,
>
> I have the perspective of a Java developer using the Java BDXML  
> API, where the implementation details of Berkeley DB are hidden  
> from me, so I don't have a good understanding of when exactly bdxml  
> locks/unlocks and what the difference is between a "lock", a  
> "locker" or a "lock object" and also the difference between a  
> "deadlock" and "conflict".
>
> I found out that this particular code fragment causes a self- 
> deadlock(even though it detects as a conflict!):
> XmlQueryContext qc = theManager.createQueryContext();
> XmlModify mod = theManager.createModify();
> ...
>   XmlResults allDocsSel = theManager.query(xQueryDocSelection,qc);
>   XmlUpdateContext uc = theManager.createUpdateContext();
>     mod.execute(allDocsSel);
> allDocsSel.delete();
>   uc.delete();

I believe that the self-deadlock in this case may be a problem in BDB  
XML2.1.8,
but as you are aware, you and I are still working on that.

>
> When I instruct the query context to use live data with lazy  
> evaluation, and iterate over each value in XmlResults, the self- 
> deadlock is avoided. I also found out that the "delete" statements  
> were not necessary to avoid deadlock.
>
> XmlQueryContext qc = theManager.createQueryContext 
> (XmlQueryContext.LiveValues,XmlQueryContext.Lazy);
> XmlModify mod = theManager.createModify();
> ...
>   XmlResults allDocsSel = theManager.query(xQueryDocSelection,qc);
>   XmlUpdateContext uc = theManager.createUpdateContext();
>   XmlValue thisValue = null;
>   while((thisValue = allDocsSel.next())!=null)
>   {
>    mod.execute(thisValue,qc,uc);
>    thisValue.delete();
>   }
>     allDocsSel.delete();
>   uc.delete();

The difference you are seeing is related to the use of Lazy evaluation.
When locking is configured, and transactions are not being used,
locks are held on documents.   One or more of those locks is what
is causing the deadlock.   Lazy evaluation defers the acquisition of the
documents, and the locks, which solves your problem.

In general, however, the locks should not get in the way, even
without lazy evaluation.

>
> I've synchronized my methods to further prevent any kind of  
> internal deadlock and it appears to function until I seem to run  
> out of locker entries!

Using locking without transactions, and still using multi-threaded  
access to
modify documents, you definitely need to synchronize.
However, if you are synchronizing in that way, you probably do not
need to configure locking at all, unless there are other processes  
accessing
the containers.

>
> How do I reclaim locker entries/locks/lock objects? I assumed they  
> would "free themselves up" after a certain period of time. I've  
> read that  XmlManager and XmlContainer hold locks, so I closed and  
> deleted them after every use, but the locks are still being used up...

You should not delete XmlManager and XmlContainer objects frequently.
That will cause performance problems.  The locks they hold are
insignificant.

The locks that matter are those held per-document.  In order to release
them, make sure to delete these objects after use:
     XmlDocument
     XmlValue
     XmlResults

If you are doing this, and still running out of resources, then we  
need to dig
deeper into the reasons.

Regards,

George

>