Re: wget 1.10.x: New option --random to randomize the dns result

Hrvoje Niksic wrote:
> Jerry Lundström <[email protected]> writes:
> 
>>As getaddrinfo() sorts the dns result no matter if its ipv4 or ipv6
>>you will be resolved to the same server 99.9% of the time. I wished
>>there was a flag to getaddrinfo() so that you simply could turn off
>>the RFC 3484 behaviour but there isnt.
> 
> I understand what the option does, but I don't understand what value
> it brings to the end user of Wget.  Wget is already capable of
> reconnecting to the next address in the list if the first one fails
> (and not use the one that failed during the same Wget run).  I'm
> trying to imagine which users would ever want to use this option and
> on what grounds, and I can't come up with a plausible scenario.

A user scenario could be that wget with ipv6 enabled always picks a
broken website since RFC 3484 deside that it should sort IPs in the way
it does. For a novice user that might be hard to understand that its
just one mirror/roundrobin of the url thats broke and not the main.

It is fatal to automated script to always pick the broken mirror.

Like I said before, I would have loved a flag to getaddrinfo() to simply
just not sort by RFC 3484, that would have been a much better solution
to use the dns information as its given by the dns resolv.

Also, as mirror manager you might get 1/2 overloaded servers since the
sort function is the way it is. We have mirrors that are hardly used and
its no wonder if their ip never get selected.

>>Mirroring scripts, dists and other things that use wget might not
>>always like getting the same ip address over and over again.  I am a
>>Lunar Linux developer (www.lunar-linux.org) and we use wget for
>>fetching sources and RFC 3484 breaks our mirrors before they are
>>synced.
> 
> Do you mean that the first address in the list is malfunctioning?
> Shouldn't Wget then fall back to the next one?  If one of the
> addresses is malfunctioning, even with --random you could still get
> that address, right?

I don't see this behavior at all, if the first address returns 404/403
it will not move on to the next.

As it states in the --help, wget doesnt retry on 4xx's altho it has
--retry-connrefused . But if a mirror returns 404/403 it won't try the next.

Ofcourse --random wouldn't solve 404/403's but it would not give the
same address everytime.

-- 
Jerry Lundström, System Developer
The Division of IT and media, Stockholm University, Sweden
+46 (0)8 16 19 99 / http://www.it.su.se