Re: Request-For-Ideas: requires statistics and thoughts on making our metadata smaller

On Nov 2, 2009, at 10:09 AM, Jeff Johnson wrote:
>
> If anything, there are more redundant strings in the
> XML markup than the dependency content itself in rpm-metadata.
> But that flaw is usually dismissed with
> 	Comress! Compress! Compress!
>

I should point out that XML has the ability to do memoization
as well, using tags-by-reference instead of strings-by-value.

But the markup quickly becomes unreadable, and one
_MUST_ carry around the memozation "dictionary" and provide
tools to read the markup. Equally true for compression,
*.solv, and sqlite, all of the redundancy removal of
each of those forces a decoding will converting data
back to something that is readable. They approaches
to removing redundancy differ solely in what tools are used,
all are removing redundancy with "dictionary" lookup on
redundant strings.

73 de Jeff