Re: Transaction Set clean does not actually clean

On Dec 14, 2007, at 2:08 PM, Mitko Haralanov wrote:

> On Fri, 7 Dec 2007 15:59:29 -0800
> Mitko Haralanov <[email protected]> wrote:
>
>> That's it! I can use the wrapper for thread safety, keep the ts
>> instance around and intercept the ts.clean() calls.
>
> The idea that you offered works perfectly when trying to clean the
> transaction set.

Good.

> However, now I am running against a different problem which I can't
> figure out how to solve without opening and closing the database:
> Since my application act like a daemon, waiting for rpm requests, the
> TransactionSet (when created) locks the database and does not allow  
> any
> other application to work with it.

Opening & closing an rpmdb is like killing mosquitoes with a bazooka.

> This, of course, is solvable with opening and closing the database but
> I remember you mentioning that this should not be done because there
> are other things that should be taken care of if done.
>

OTOH, if your goal is a dead mosquito, then just make sure that
you also have voided out any retrieved information, particularly
internal information like void keys.

Yum script kiddies are still waiting to learn that lesson afaik ...

> Would you mind elaborating on that?

You are likely dead locked in one of 3 ways:

1) The layer of single threading you've used in python on top
of (not thread safe) rpmlib. Presumably you can sort that problem  
pretty easily.

2) The fcntl lock used to protect transactions. That lock blocks
readers iff rpmtsRun() (or --rebuilddb) is called and the lock  
becomes exclusive.

Use lsof to verify whether the file path used by the transaction lock  
is open.

3) The Berkeley DB INIT_CDB cursor locks on rpmdb access itself.

INIT_CDB is used for concurrent access protection with a guarantee  
that there
is only a single write or multiple readers of a Berkeley DB at any  
point in time.

You're likely dead-locked because of the INIT_CDB locking, its the  
hardest to
diagnose problems with.

Display the current locks in a dbenv using db_stat from Berkeley DB  
utilities.
There's a copy of db_stat in /usr/lib/rpm, often renamed to rpmdb_stat.

E.g. here's a shared read lock from a paused "rpm -qa"

$ sudo /usr/lib/rpm/db_stat -Cl
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock REGINFO information:
Lock    Region type
4       Region ID
__db.004        Region name
0xb7c3d000      Original region address
0xb7c3d000      Region address
0xb7c3d044      Region primary address
0       Region maximum allocation
0       Region allocated
Region allocations: 3005 allocations, 0 failures, 0 frees, 1 longest
REGION_JOIN_OK  Region flags
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by lockers:
Locker   Mode      Count Status  ----------------- Object  
---------------
[jbj@wellfleet rpm]$ sudo /usr/lib/rpm/db_stat -Cl
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Lock REGINFO information:
Lock    Region type
4       Region ID
__db.004        Region name
0xb7c12000      Original region address
0xb7c12000      Region address
0xb7c12044      Region primary address
0       Region maximum allocation
0       Region allocated
Region allocations: 3005 allocations, 0 failures, 0 frees, 1 longest
REGION_JOIN_OK  Region flags
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Locks grouped by lockers:
Locker   Mode      Count Status  ----------------- Object  
---------------
        4 dd= 0 locks held 1    write locks 0    pid/thread  
497/3086265024
        4 READ          1 HELD    (30769 802 cdc6e56d 1056 0)  
handle        0
        5 dd= 0 locks held 1    write locks 0    pid/thread  
497/3086265024
        5 READ          1 HELD    0x3a134 len:  20 data:  
i0x070x03000x020x080000m0xe50xc60xcdV0x10000000000000
        6 dd= 0 locks held 1    write locks 0    pid/thread  
497/3086265024
        6 READ          1 HELD    (30783 802 e3d3478b 1b8796 0)  
handle        0

Note the process and thread id's that are displayed.

That's the starting point of the diagnosis. My guess is that one
of your threads has left an iterator open somewhere.

Note "guess" please ;-)

73 de jeff