Re: [PATCH 4 of 4] hard link packages between local caches

On Wed, May 30, 2007 at 05:36:03PM -0400, seth vidal wrote:
> On Wed, 2007-05-30 at 19:15 +0100, Daniel P. Berrange wrote:
> > When syncing multiple repositories across many architectures there will be
> > a fairly large number of duplicate RPMs present in all repositories. This
> > is particularly true of x86_64 multilib distros which share most of the
> > i386 packages. Downloading many GB of packages for x86_64 repo which are
> > already present in a i386 synced repo is wasteful of time, disk space and
> > bandwidth. So this patch adds an extra argument --package-cache which takes
> > a directory path. This argument can be repeated multiple times. Before 
> > downloading a package, reposync will check each of these directories to see
> > if they already contain a copy of the package. If so, the local package will
> > be hard-linked into the destdir.
> > 
> > Typical usage would be to list the i386 repo cache, when running a sync of
> > the x86_64 repository, and vica-verca.
> > 
> > To give an idea of the saving, Fedora rawhide currently has ~7400 rpms in
> > i386 repos, and 13,000 in x86_64 trees. ~6900 of these RPMs where present
> > in both trees. Hardlinking saved 8 GB of disk space & avoid 8 GB of file
> > downloads :-)
> > 
> 
> This patch looks good, but I'd be happier if you would check the
> checksum from the package to make sure it matches the remote metadata
> instead of just relying on name, size, etc.

No problem - I'll knock up a revised patch doing that tomorrow. I'll also
make it do the check for existing RPMs in the destdir too since that really
has the same integrity needs too - particularly if YUM does continuation
of interrupted downloads.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|