Re: Problems importing huge amounts of data

On Friday 03 Mar 2006 10:32, Santi Camps wrote:

> Then we began to debug the DS itself, and found that the "problem" was
> that DS was reading all the files during the commit process (more than
> 30000, because there is a .c and .oid file per object),    Obviously,
> it's not an error, but it has a tremendous cost.

Interesting. What percentage difference did this make to performance?

> So, we patch the DS in order not to do this readings when parameter
> sync = 0.    I attach the patch, hopping it could be included in next
> releases

This reading is part of a mechanism to prevent 'dangling references' in the 
object database. You can suppress this whole mechanism by changing 

check_dangling_references: 1

to 

check_dangling_references: 0

in your configuration file. This should eliminate the same disk access as your 
patch, plus some other CPU time.

-- 
Toby Dickenson


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642