Re: metadata layout problems and some history

On Sat, 2010-08-07 at 17:37 +0200, Michael Schroeder wrote:
> So it takes about a second to convert the 2.8 Mbytes xml file to
> solv. I guess creating the sqlite database is a bit slower, so you
> chose to do it on the server and not on the client.

As the size of this file increases how much time is eaten up? You showed
a f12 updates repo, it seems.

Fedora 13 GA for x86_64 - the compressed primary.xml.gz is 7.9M.

So does that mean the solv-creation is going to take ~3s?

I don't see the number of pkgs decreasing at any point in time and since
we were pushing fedora onto some REALLY underpowered boxes we found the
xml->sqlite conversion can SUCK on some machines (OLPC XO-1's for
example were horrendous)



> As parsing the data file is not the bottleneck for us, anything that
> reduces the transfer size is a good thing. A switch to sqlite
> would hurt us, as the sqlite database is currently bigger than the
> compressed xml. (This is probably worse if lzma is used
> instead of gzip.)

Do you end up storing the xml AND the solv files in the cache dir? I ask
b/c this was another reason for doing server-side-generated sqlite b/c
we were keeping both in the cache and, again, on the XO-1 and other
quasi-small-disk systems (and most importantly SSDs) we were trying to
keep the disk writes and disk use as low as possible.


> 
> (I have no e-mail access for the next two weeks. Klaus and Duncan,
> could you please answer any questions?)

Ahh - a wish to have no email for two weeks. :)

Enjoy your vacation.

-sv