Re: Mirrorbrain demo instance for debian-cd?

On 05/21/15 10:16, Joerg Jaspert wrote:
> Sorry for the delay in answering this, I was kind of hoping that one of
> the other mirroradmins pick it up. :/
> I do agree that $something nicer than the current page sure makes sense,
> and would be happy to help get it into shape to eventually move to a
> debian.org machine.
> I think it should start similar to httpredir - that is, run elsewhere
> with a debian.net domain for a while, and when it works out it gets
> moved. That also gives time to get the softwarepackaged properly.

I also think that would be the way to go.
There may be some pitfalls that only show up after some time, and it
would really be a shame if that happens after a lot of time has been
spent on getting everything perfectly into debian infrastructure.

> Now, on that: I'm happy to use my DD hat for sponsoring/mentoring a
> package upload, getting the debian.net domain, use my mirror hat to give
> input for the new instance, ... - but I won't be the one doing the
> (main) work. Simply no time for it.
> So if you, despite my late answer on this, are still prepared to do it,
> are you prepared to go the full way, *possibly* ending up a DD (or at
> minimum, DM for uploading mirrorbrain) and member of mirrors@
> yourself? :)

I would still be willing to run and maintain a demo instance under a
debian.net domain, in close cooperation with the mirrors-team. This
would naturally involve keeping the list of mirrors in it up to date
(that should be easily automated from Mirrors.masterlist), providing
feedback on problems encountered, and investigating problems reported
with the redirector as far as possible (although I suspect most issues
reported would be bugs that are completely irreproducible, e.g. because
some weird "http optimizer" handling the free wifi at a certain
starbucks branch messed up all http headers). I would also help to get
this moved to proper DSA-maintained infrastructure one day.

However, getting the (existing) mirrorbain-packages ready for upload
into the official debian archives is not my cup of tea. Packages exist,
they work, all that is left is to make them debian policy compliant, and
that would mostly be a job for some experienced package maintainer. And
I do not intend to become that deeply involved with debian packaging.
Maybe Raphael H., who essentially already volunteered in
http://mirrorbrain.org/issues/issue26, or Andrea Veri who idles in the
#mirrorbrain channel, could be persuaded? Of course that only really
needs to be decided after it has been proven that mirrorbrain really
works well for debian-cd.

>> There is however existing software that does exactly what is needed for
>> debian-cd, a very popular one is Mirrorbrain [2].
> It's popular - does it mean its the best? What others are out there, did
> anyone look and/or compare? I'm offline in train currently, so unable to
> look for myself.

I must admit that I did not thoroughly research this, as I was
personally mostly interested in how mirrorbrain works. And the list of
features mirrorbrain has really is impressive. It gives you extremely
fine-grained control about what you want to redirect where. For example,
you can set the percentage of requests a mirror gets, to adjust for
mirrors with limited vs. huge available bandwidth, or choose not to
direct some files at all (because they are so small that it makes no
sense, or because they contain critical info like hashes). In addition
it supports other nice-to-haves like generation of .torrent files that
contain nearby mirrors as webseeds.

To the best of my knowledge Mirrorbrain is the only software of its kind
that
a) was not hacked by one specific project for it's own needs (e.g.
raspbmc/osmc uses something homebrewn), and most of the time never published
b) does not depend on a specific structure of the tree being served,
e.g. httpredir only works for debian archive trees.

I'm not sure about Fedoras Mirrormanager. It does also handle their .iso
downloads, but I'm not sure how flexible it really is w.r.t. directory
structures - their documentation seems a bit lacking. At Fedora you can
also - as a mirror admin - submit/disable/remove/manage your mirrors
yourself through their admin interface. You may specifiy ASN and subnets
you are serving and clients from these nets will be redirected to you
exclusively. That simplified handling of mirror infrastructure might
really be nice - if it's flexible enough to work with arbitrary file
trees. But I don't think it is - looking into that webinterface, it
gives me a list of directories that it thinks are up to date on my
mirror. If it works on a directory level as that suggests, then it's not
really ideal. For example, a mirror might choose to only offer
DVD-1.iso, because it is low on disk space and that will be by far the
most requested file. With DVD-2 and DVD-3.iso being in the same
subdirectory, if you look at it at the directory level, that mirror
would never be considered 'current'. For Mirrorbrain that situation
would pose absolutely no problem.
I think Raphael G. had a closer look at mirrormanager before doing
httpredir - maybe he can shed some light onto what it can and cannot do?

One final somewhat negative point should be mentioned about mirrorbrain
though: mirrorbrains main developer nowadays seems to have left IT, that
means he is no longer paid to develop mirrorbrain because his employer
runs it. You can see that from the release history, new versions
nowadays are scarce. However I do not consider this to be a mayor
problem - it is pretty featurecomplete, and judging from his posts on
twitter, he still considers Mirrorbrain his 'baby', which makes me think
that at least security issues would be addressed promptly (there haven't
really been any in the project history).