Re: pre-PEP: transition to release-file hosting at pypi site

On Mar 11, 2013, at 4:07 PM, Carl Meyer <[email protected]> wrote:

> On 03/11/2013 01:57 PM, PJ Eby wrote:
>> I'm saying that if someone objects to the presence of  links they
>> don't actually use, they are speaking nonsense.  Might as well ask to
>> ban all packages from PyPI that they don't personally like -- it's the
>> same request.  Nobody is forcing you to depend on packages that don't
>> host on PyPI, so there is no point to the censorship.
>> 
>> If you don't use the links, you can't argue that their presence is
>> causing you harm.
> 
> You can, of course, argue that the mere presence of those links
> (combined with the current behavior of easy_install/pip) is an
> "attractive nuisance" that indirectly causes harm to unsuspecting new
> users of Python who never even consider the possibility that tools like
> easy_install and pip might spider off PyPI to arbitrary websites (a
> reasonable assumption based on experience with automatic installation
> toolchains and software repositories in other communities). I've talked
> to many such users, so there is no question that they exist, and I think
> probably in significant numbers.
> 
> Carl
> _______________________________________________
> Catalog-SIG mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/catalog-sig

Since it was asked I had ran a script to see which projects/versions that my earlier script had identified as not being hosted on PyPI to determine _where_ people are hosting these files. These statistics include dev releases.

There are 10538 total external file links that pip locates that do not exist on PyPI.

Of these here is the top 20:

    (u'downloads.tryton.org', 1201),
    (u'github.com', 811),
    (u'bitbucket.org', 428),
    (u'launchpad.net', 279),
    (u'www.doughellmann.com', 255),
    (u'walco.n--tree.net', 161),
    (u'prdownloads.sourceforge.net', 156),
    (u'infrae.com', 150),
    (u'downloads.sourceforge.net', 139),
    (u'keepnote.org', 138),
    (u'downloads.reviewboard.org', 124),
    (u'tilestache.org', 121),
    (u'mercurial.selenic.com', 120),
    (u'www.defuze.org', 85),
    (u'www.vicbioinformatics.com', 74),
    (u'downloads.review-board.org', 70),
    (u'samba.org', 70),
    (u'python-graph.googlecode.com', 67),
    (u'cyberelk.net', 65),
    (u'tuohela.net', 61),

I suspect that a lot of the github, bitbucket etc links are dev links (of which there are roughly 420 total).

Here is the complete listing: https://gist.github.com/dstufft/5137885

I ran a minor bit of heuristics to see how many were not hosted in one of the big name hosting sites,

>>> sum([x[1] for x in b if not "github.com" in x[0] and "bitbucket.org" not in x[0] and "google" not in x[0] and "sourceforge" not in x[0]])
7097

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

_______________________________________________
Catalog-SIG mailing list
[email protected]
http://mail.python.org/mailman/listinfo/catalog-sig

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)

iQIcBAEBAgAGBQJRPkwPAAoJEG48vOkzctz634gQALTQt2GEzvAV8iNhYnNCgSI1
Vyr2bkISe6O5ZcPaRezHUigT8EgFfzvSnUJinp5hzMQqhva1eKlXqCUTtlSzPJ5k
ugeNOdbY4mZvp7bT+jrIo+YnrJkkT2Ap6osLTqzj3PmyId8v5WEA7LfetjyLu9y0
KxTy0wDgoQMB3oQfENem600Z9yYXfqFPcRwn6k3awOMydG1saS3u0tcS4XN+aCla
Rcclds8H4d285PBrvfIgN/CAPiSc2D5dgGv7nwMbAw9KP0C7phrjj75kuH8jFzjc
1xkKE5FEqniyvWyv20vGZLiG4mLqUj9HdeFo6pV34z6+Hm21rTL+4N6AM5U1YFDE
nhOI+qE+EszFZ/yECA9Kb4639Y/Eo3FBDeQUH3L+LWcI/ITQ/Mmmp0jqkSe6aNuL
h6Eb5bTg0k5Llm5QZMt/mN8+4nV7PVifFbWcJS1Mgb3esVXMgUvOboJs8O5SPcrX
9yUx3s/r0088qQry5EF5dkgMGk3IT8rNz8VjT6AhfC4zMRDTj9GVHpYIkeAvHL+k
u7ffMKth9zLfLDBILbAS5LxHSTgu1KaCauSvLFCwWt8GkpgQcv5gWBImT6ga94UE
cAjOXwobfp+KJBBQmjTX5k9Do+VC0gfmTsu8menn33vOh7e3i25iykyn9GbukqDI
a6SvX9jTq2yMfo/CyDh9
=fTxB
-----END PGP SIGNATURE-----