Mirror layout proposal

Graham Forest <[email protected]> 01 Aug 2003 12:22:03 -0700
Newsgroups gmane.linux.zynot.zynaut
Message-ID <[email protected]>
--===============69007493488268157==
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature";
	boundary="=-GgruYnlfVTq4NN1Eb5A1"


--=-GgruYnlfVTq4NN1Eb5A1
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

So, the time has come to take a first jab at how to layout mirrors.
There are, in my opinion, a few basic requirements which I'll get out of
the way first:

* Identical paths on all mirrors
    Identical paths are necessary for roundrobin dns, and general
sanity. zynot.oregonstate.edu has our files at /pub/zynot, so it seems
logical to me to require that path point to our distribution trees on
all mirrors. /pub may be too ftp specific, in which case possibly /zynot
would be adequate.

* Organized distfiles section
    It's my opinion that the distfiles should be kept in subcategories.
In my opinion there should be a folder per application, ie all
app-admin/foo distfiles be placed in an app-admin/foo/ subcategory. Of
course, this would require that some ebuilds that use the same distfiles
be tweaked to download such shared files into the most logical 'parent'
directory, but we're going to be thrashing the trees rather thoroughly
anyway, so that shouldn't be too much of a problem (also, it's not a
huge deal to mirror some files multiple times accidentally, and such can
be easily remedied later).

* Per-dev (or project) mirror space
    For Zynot-hosted projects, and other various things devs may need to
distribute, there needs to be a section in the tree. This would be
synced either out of ~/something on jupiter (dev shell box), or part of
the SVN tree.

* Sane, standard gpg signature distribution
    Mirrors should not have the ability to compromise our security by
changing the contents of tarballs. md5sums are not a practical method of
thwarting this, as more often than not it is assumed that the md5sum is
wrong, not the file. Users should be given the option to reject
distfiles that have not been gpg signed by X number of trusted devs, and
those signatures must exist somewhere in our distribution tree. An idea
brought up in #zynot-infra is to have the downloading of the gpg
signatures default to always coming from our main mirror, only getting
them off other mirrors after failing at that, and printing a nice
warning message. Of course, with proper key signing and distribution the
security benefits of this are reduced, but a nice side-effect is that we
would be able to track the approximate number of downloads without
heavily monitoring every mirror.

* Easy to navigate iso/stage file distribution
    I think just about everyone has spent a few hours of their life
hunting around badly layed out ftp servers. Users should be able to
click to direcly what they want without having to search. In my mind,
this means not only having an intuitive layout, but also providing handy
things such as links to "current" versions (when possible, symlinks may
cause problems).


So, I throw the following out to be poked and prodded until somewhat
sane:

/pub/zynot/
    distfiles/			# Organized distfiles dir
        app-admin/
            foo/
                foo.tar.bz2
                foo.tar.bz2.asc	# Signatures (hopefully more than one=20
				# per file)
                ...
            moo/
                ...
            ...
        app-benchmarks/
            bar/
                bar.tar.bz2
                bar.tar.bz2.asc
                ...
            ...
        ...
    people/
        gforest/		# Synced from
/svn/zynot/people/gforest/something?       =20
    releases/
	stable/			# Symlink?
	unstable/		# Symlink?
        0.1
            x86/
                iso/
                    zynot-moo-foo-ra-gu-0.1.iso.bz2
                stages/
                    zynot-ug-ar-oof-oom-0.1.tar.bz2
            ppc/
                iso/
                    zynot-moo-foo-ra-gu-0.1.iso.bz2
                stages/
                    zynot-ug-ar-oof-oom-0.1.tar.bz2
            ppc64/
            ...
        1.1_rc385		# Just kidding
            x86/
            ppc/
            ppc64/
            ...
        ...
    ...

I appreciate all prompt constructive feedback.

Have fun but remain pleasant,
Graham

--=-GgruYnlfVTq4NN1Eb5A1
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD4DBQA/Kr1BAuxgtoh9rPYRAp0LAJdlDhFBfZ2IENvkDl8uo/foxx46AJ9F4KGm
BS9XzlCXb6urqLDC/IQc8g==
=2D/W
-----END PGP SIGNATURE-----

--=-GgruYnlfVTq4NN1Eb5A1--


--===============69007493488268157==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Zynaut mailing list
[email protected]
http://lists.zynot.org/mailman/listinfo/zynaut

--===============69007493488268157==--