Re: V4 Pre-PEP: transition to release-file hosting on PYPI

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--===============1140713509==
Content-Type: multipart/signed; micalg=pgp-sha1;
 protocol="application/pgp-signature";
 boundary="------------enigAD2862FFBA59478E0FE9AF33"

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enigAD2862FFBA59478E0FE9AF33
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

tl;dr: I see your points, we'll change the PEP to allow clients to use
hostnames instead of the rel attributes if they prefer. More comments bel=
ow:

On 03/15/2013 12:59 PM, PJ Eby wrote:
> That's the bit I don't like.  The security model is that if it's not
> allowed by allowed-hosts, it's *not allowed*.  Introducing a way to
> sneak something past allow-hosts is a bad idea, because it means
> people either have to explicitly widen their allow-hosts to arbitrary
> hosts, or else that you can't actually enforce an allowed-hosts
> policy, or that you need to learn a whole bunch of options to
> implement it.
>=20
> ISTM that this is a bad design choice for users, and I'm not
> comfortable with this without some way to define the allowed
> "internal" hosts based in some way on the base index URL.  Not just
> for ease of automated translation, but so that *users* can know who
> they're dealing with, and easily predict the effects of their chosen
> options.
>=20
> A frequent refrain has been, "users don't know they're downloading
> stuff from places other than PyPI", so if this new approach allows
> downloads from somewhere other than *.pypi.python.org when you've
> chosen pypi.python.org as your index, ISTM the proposal is failing to
> meet its original goals.  As the PEP is written, PyPI could change out
> to a different CDN each week or use different ones for different
> files, and users would be back in the position of not being sure where
> stuff is coming from.

I guess the key question is the definition of "places other than PyPI."
I think a CDN that is part of the index's architecture is just as much
"part of PyPI" whether it's on the same domain or not. But I understand
the difficulty integrating this with the --allow-hosts option in a way
that maintains a clear and simple UI.

> I'm fine with extending the default host matching to
> "indexhost,*.indexhost" if we want to leave more of an option for PyPI
> and other indexes to use a CDN.  But I'm not sure how much point to it
> there is, since a /simple index is static, and small in size compared
> to the downloads, so you might as well host a copy of the /simple
> index alongside the downloads, and make the index pypicdn.com/simple
> or whatever in the first place.  (In other words, not a lot of benefit
> to splitting a static index from its associated files, so why support
> it?)

Putting the /simple/ API on a CDN isn't quite that easy because it
currently involves some server-side redirects to effectively make
project names case-insensitive. I think in a hypothetical
re-architecture of PyPI there may be good security reasons to put
user-uploaded files on a different domain from dynamic portions of the
API (Donald alluded to this, more discussion at
http://security.stackexchange.com/questions/11756/is-it-safe-to-serve-any=
-user-uploaded-file-under-only-white-listed-mime-content).

So I think this issue may come up again in the future. But I'm fine with
deferring it in this PEP for now...

>> PyPI wouldn't be enforcing a UI on you here, just providing metadata
>> that you can use as you wish.
>=20
> That's not what the PEP says.  It does in fact *mandate* the use of
> the rel attributes.  So if somebody adds an "external link" that
> actually points back to PyPI, technically I'm not supposed to use it
> unless it's been explicitly authorized.  ;-)
>=20
> I'd really prefer to see explicit language that says the rel
> information is advisory only and that installers aren't required to
> parse it, let alone use it.  At the moment, the PEP is a substantial
> departure from the version I agreed with.

Ok, pending agreement from Holger I'll make a change in the PEP to
explicitly allow clients to make decisions based on either the rel
attributes or based on hostnames. Would that be sufficient to address
your concerns?

Carl


--------------enigAD2862FFBA59478E0FE9AF33
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlFDq8cACgkQ8W4rlRKtE2e7DwCgt3vhgakoBOSJgr9ycRH4URbb
yzQAoKIHEOhw9rbkxFbIhG8fxIIIohQW
=dkfP
-----END PGP SIGNATURE-----

--------------enigAD2862FFBA59478E0FE9AF33--

--===============1140713509==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Catalog-SIG mailing list
[email protected]
http://mail.python.org/mailman/listinfo/catalog-sig

--===============1140713509==--