Re: discussion of createrepo and repodata format future

On Fri, 2010-08-06 at 14:55 +0200, Klaus Kaempf wrote:
> * seth vidal <[email protected]> [Aug 06. 2010 14:34]:
> > 
> > When you think about it - the xml format is a cache, too. It's a
> > compilation, truncation and compression of the data in the rpm hdrs.
> > 
> 
> Actually, it serves two purposes (requirements)
> 
> 1. caching of rpm header information
> 2. providing a universally understood format ("XML's design goals
>    emphasize simplicity, generality, and usability over the Internet"
>    (according to http://en.wikipedia.org/wiki/XML)

 XML is no more "universal" than sqlite, or a few other formats.

> I'd like to add a third requirement
> 
> 3. minimize download time required by clients to update their knowledge
>    about a repository

 That's fair, but there are several points to this:

i. How big are the metadata file sizes.

ii. How much of it you have to download for several usecases (install,
search, provides).

iii. How much of it changes, as packages are added. Some kind of delta
could help here, as could changing the format (a bit, or a lot).

iv. How much work the client has to do post download.

v. How much work the client has to do when the metadata changes.

vi. How big "repomd.xml" becomes.

...because users see the sum of all of this as "download time", in most
cases.
 Adding ".xz" compression trades #i for #vi, and is probably worth it.
Using XML trades #i for #iv, #v and #vi ... and IMNSHO, isn't worth it.

 The start of this thread, is due to #ii currently being far from
optimal for some cases.