Re: download strategy, was Re: apt, yum, up2date, smartpm, ... ?

Good evening, Les,
 	Thanks for your ideas!

On Wed, 21 Sep 2005, Les Mikesell wrote:

> On Wed, 2005-09-21 at 17:19, William Stearns wrote:
>
>>  	For the repositories and distributions you request it defaults
>> to getting all of their files.  That said, you can:
>>
>> - place any files you already have in the tree in advance so they're not
>> downloaded again
>
> Is there an automated way to do this for the base distribution or
> pretend you've done it by loopback mounting the distribution
> iso images?

 	The question we're trying to address is how to handle low 
bandwidth lines efficiently; people with this problem are less likely to 
have access to the iso images for these distributions.
 	In the cases where the user does have access to the iso images, 
the files can be copied over with any file management tool.  Loopback 
mounting the images means you still use the space for the iso files, but 
saves about $4 of disk space by not copying the rpms.
 	If that's important to you, yam does allow loopback mounts of iso 
images.  yum-pull is able to use loopback mounted files, it just doesn't 
do the mounting and symlinks.

>>  	If it's simply a matter of wanting to avoid duplicate downloads,
>> it would seem that using proxy such as squid would avoid duplicate
>> downloads from a given internet server; the "proxy" option in the "[main]"
>> section of /etc/yum.conf would allow you to specify the location of the
>> squid proxy to use (man yum.conf).
>
> This doesn't work for repositories that use the mirrorlist option. Yum
> picks something different from the list every time, cluttering the
> cache and defeating its purpose.

 	Good point; I hadn't thought of that.  You'd have to put in a 
restricted list of mirrors to have this help at all, and cut down to 1 
mirror to get full use of the cache.

>> modem-based users.  Is there a third option that's an alternative to "get
>> all files in advance" and "wait for the files to come down after the user
>> requests a package install or upgrade"?
>>  	Would "download only new versions of packages I currently have
>> installed" be the third option you want?  That would speed up patches, but
>> wouldn't help with installing new software and might run into problems if
>> a patch rpm requires libraries you don't currently have installed.
>
> No, it would need to work like a proxy cache combined with

 	(a role squid could handle well)

> yum --download-only ... &
> so yum could decide what it needs and get it in the background or
> as a scheduled command - and this needs to report it's status so
> you know whether or not everything is available for the subsequent
> real yum action.

 	Good approach.
 	One minor detail is that the --download-only option was removed 
from recent versions of yum.  Having one machine do its download in the 
middle of the night would pull down the files the other machines would 
later need.  You could even nfs mount the lead machine's /var/cache/yum 
over to the other hosts and run them with "yum -C".

> There is another reason for wanting to download a local copy of a
> repository, though.  If you want to update one machine, then do
> some testing, then update some others to match what you tested
> there is no way to tell yum not to take newer packages that have
> become available in the repositories to make the updates repeatable.
> Will yum-pull handle that scenario?

 	Actually, it will.
 	In the instructions at 
http://www.stearns.org/yum-pull/yum-pull-setup.html , I cover how to set 
up a staging server for people that want to test rpms before distributing 
them to clients.
 	Cheers,
 	- Bill

---------------------------------------------------------------------------
 	"I don't have any solution but I certainly admire the problem."
 	- Ashleigh Brilliant
(Courtesy of Steve Dodd <[email protected]>)
--------------------------------------------------------------------------
William Stearns ([email protected]).  Mason, Buildkernel, freedups, p0f,
rsync-backup, ssh-keyinstall, dns-check, more at:   http://www.stearns.org
--------------------------------------------------------------------------