Re: discussion of createrepo and repodata format future

On Fri, 6 Aug 2010, seth vidal wrote:

> On Fri, 2010-08-06 at 12:20 +0200, Duncan Mac-Vicar P. wrote:
>> I was thinking about the sqlite part, and I think the change has to go
>> in a way that
>> * assumes sqlite3 is _yum_ cache
>
> why? Why not just a random-access md?

/me wakes up from hibernation to dust 10cm layer of dust off my apt-rpm hat

Because yum is the only depsolver that can use it directly. Everything 
else uses their other internal cache formats, requiring yet another 
conversion on the data. And for a one-time read-through + convert of data, 
sequential read of xml is/can be actually faster than random access reads 
from sqlite:

On my laptop, apt-rpm with F13 default repositories, generating the 
internal cache from scratch takes ~35s with XML files. With the sqlite DB 
files, it takes ~46s. From what I read here, zypp and smart share similar 
experiences (whatever the exact numbers are I've no idea).

In case of apt-rpm, sqlite /is/ a huge win over XML for the operations (eg 
file searches) where the data isn't stored in the internal cache but has 
to be looked up from the repository files. With XML those operations are 
simply pathological, similar to the smartpm case of 30min vs 30s vs 3s.

If we're talking about redesigning repomd, it'd be a huge mistake to not 
at least attempt to address all the (now) known problems of the initial 
design + extensions it has grown over time, and one of them is: what's 
good for yum can hurt others, because they operate in a wildly different 
ways. And mind you, I'm not pointing any fingers as I'd be as guilty as 
anyone who's been around since the initial repomd spec discussions and 
either not seeing the issues coming or didn't speak up / submit code.

 	- Panu -