Re: speeding repository rebuild (compared to yum)

On Tue, 2004-09-07 at 19:51, Marc MERLIN wrote:
> Hi,
> 
> Having been a debian user for a while, when apt-rpm came out, it was the
> obvious choice for me when I built a repository (and yum was still new back
> then)
> 
> My problem is that I now have 600+ contrib RPMs that I maintain in
> a tree, and every time I add one, the rebuild takes 20-30mn (which is not
> as bad as rebuilding the RH9 pository when they were still coming out with
> updates, as that rebuild would take about 1H with the 1000+ RPMs)
> 
> My understanding is that the index rebuild opens up all the RPMs, and
> computes all the dependencies then, which is why it takes that long.

No, genbasedir doesn't calculate any dependencies, it merely dumps them
to the metadata files much like yum does.

> However, I've also been told that yum extract the metadata and puts it into
> a different file, allowing clients to download that and do the dependency
> processing on their side.
> Considering that a yum repository update then runs almost instantly (i.e for
> as long as it takes to take out the metadata out of whatever new RPMs you
> installed), that's quite appealing.

What genbasedir does is very close to what yum-arch does with two
significant differences performance-wise:
- apt's metadata is compressed with bzip2 which is quite heavy
computionally (compared to yum using gzip)
- gen*list utilities calculate md5sum's for each package processed,
which takes a lot of time unless the results are cached

See
http://distro2.conectiva.com.br/pipermail/apt-rpm/2004-April/002311.html
for some figures how much difference those two things can make and how
to optimize them.

> 
> Right now, I'm running genbasedir as such:
> genbasedir --flat --bloat --bz2only `pwd` RPMS.glibc2.0 RPMS.glibc2.2  RPMS.noarch RPMS.glibc2.1  RPMS.glibc2.3  RPMS.RH2.1AS
> and I use apt-0.5.5cnc5-fr2
> 
> I've seriously considered switching to yum, but I do automated rpm installs
> and updates from unattended cron, and I absolutely need 
> apt-get install --trivial-only
> and
> apt-get upgrade (and not dist-upgrade)
> In other words, I need installs and upgrades with the absolute guarantee
> that no extra package will be installed, or nothing will be removed as a
> result. As far as I can tell, yum does not provide that functionality.

Yum does have similar operations as upgrade vs dist-upgrade (update vs
upgrade in yum) but not the concept of --trivial-only IIRC.

	- Panu -