Re: syncrepl_del_nonpresent and syncrepl issues.

--On May 13, 2010 2:23:33 PM +0900 Jorgen Lundman <[email protected]> 
wrote:

>
>
>>>
>>> dbconfig set_lk_detect DB_LOCK_DEFAULT
>>> dbconfig set_lg_max 52428800
>>> dbconfig set_cachesize 0 67108864 1
>>> dbconfig set_flags db_log_autoremove
>>> dbconfig set_lk_max_objects 1500
>>> dbconfig set_lk_max_locks 1500
>>> dbconfig set_lk_max_lockers 1500
>>
>> You don't provide relevant data on the size of your database to make any
>> comment one way or the other on whether your "set_cachesize" value is
>> correct.
>>
>> I would suggest you provide the output of du -c -h *.bdb in your
>> OpenLDAP 2.3 database directory.
>
> Coming right up! I appreciate your time in doing some sanity checks for
> me.
>
>   2.8M   DNSData.bdb
>   3.9M   DNSHostName.bdb
>   2.4M   DNSIPAddr.bdb
>   1.4M   DNSType.bdb
>   1.5M   accountStatus.bdb
>   258K   amavisSpamLover.bdb
>   642K   deliveryMode.bdb
>   1.2G   dn2id.bdb
>    62M   entryCSN.bdb
>    84M   entryUUID.bdb
>    13M   gecos.bdb
>   898K   gidNumber.bdb
>   2.5G   id2entry.bdb
>   9.0M   mail.bdb
>   7.6M   mailAlternateAddress.bdb
>   5.6M   o.bdb
>   3.9M   objectClass.bdb
>   1.0M   radiusGroupName.bdb
>    14M   uid.bdb
>    12M   uidNumber.bdb
> (no -c in Solaris, the whole dir is: )
>   9.0G   .

> Now, live slapd consumes about 800MB of memory when it is running, which
> is probably acceptable as the server is dedicated to LDAP master. (Has
> 2GB RAM).

Your server is not sufficiently powerful enough to optimally run your 
database.  Your database size is approximately 4GB in size (dn2id.bdb + 
id2entry.bdb + index databases).  For an optimal slapadd run, your BDB 
cachesize (set_cachesize from DB_CONFIG) must be able to contain this 
entire dataset in memory.  So you have only 1/2 the necessary memory to do 
an optimal slapadd load.  This is why your slapadd takes 15+ hours to run. 
For your database itself to run optimally under slapd, it needs to be able 
to hold the sum of dn2id.bdb + id2entry.bdb, or in your case, 
1.2G+2.5G=3.7GB.  So again, you do not have enough memory.  To make matters 
worse, your DB_CONFIG only allocates 64MB to slapd.  Which, quite frankly, 
is an insanely small number for a database the size you are operating.

One other tuning parameter that will help slapadd that I failed to mention 
is the tool-threads setting.  It should be set to the number of real cores 
the box has.  However, this setting will not help in your resource 
constrained environment.

The first thing you need to do is get new hardware or increase the 
available RAM in the system you have.  Since the running slapd is going to 
take the DB_CACHESIZE + thread overhead + slapd cachesizes memory, you 
probably need closer to 8GB of RAM to run your LDAP service.


--Quanah

-- 
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra ::  the leader in open source messaging and collaboration