Re: zodbupdate
Adam GROSZER <[email protected]>
| Newsgroups | gmane.comp.web.zope.devel |
|---|---|
| Message-ID | <[email protected]> |
Hello Sylvain, Wednesday, September 29, 2010, 10:36:15 AM, you wrote: >> I made a small modification to zodbupdate that makes it quite faster. >> Please give it a quick review. >> >> svn://svn.zope.org/repos/main/zodbupdate/branches/adamg-faster >> SV> I don't think your fix cover all cases. SV> Correct me if I am wrong, what you do is not unpickle class data pickle SV> unless the class_meta information pickle of that record changed, and SV> need update ? SV> However, that imply that this doesn't update anymore any class SV> reference that is not itself persistent (so is only in the SV> class data pickle) and not used by a persistent renamed-as-well class SV> (otherwise the change in the class_meta pickle would trigger SV> an update of this record). SV> If we want to make it faster, all I can see is pack the database SV> before, as it would contain less records to go through after. I am SV> not sure, that adding parallelism would help a lot, unless you have SV> the good hardware for, but we could make it to see (it is not that SV> difficult). SV> On which database size you want to make it faster ? I had 20Go SV> databases and it toke less than one hour, if I remember correctly. SV> It's long, but I don't see how to make it faster, in the code itself. On the second look I guess you're right. It just needs to unpickle everything. I removed the branch, reliability goes over speed. Would you make a release from the trunk? I added there some more logging. -- Best regards, Adam GROSZER mailto:[email protected] -- Quote of the day: A bestseller was a book which somehow sold well simply because it was selling well. - Daniel Boorstin _______________________________________________ Zope-Dev maillist - [email protected] https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )