Re: A 90% Solution

On Mar 11, 2013, at 8:12 PM, PJ Eby <[email protected]> wrote:

> On Mon, Mar 11, 2013 at 7:39 PM, Donald Stufft <[email protected]> wrote:
>> 
>> On Mar 11, 2013, at 7:04 PM, PJ Eby <[email protected]> wrote:
>> 
>>> Just a thought, but...
>>> 
>>> If 90% of PyPI projects do not have any external files to download,
>>> then, wouldn't it make sense to:
>> 
>> To be accurate it's 90% don't have any files/release available *only* externally. Most have external  files to download because it's very rare that a project doesn't include an home_page or a download_url, especially since distutils complains if you don't.
> 
> So what is the % of projects for whom the option can be disabled
> automatically, *without* disabling automated downloadability of a
> project's externally hosted files?
> 
> Your statement is confusing to me, because the having of a home page
> or download URL doesn't have anything to do with whether that page has
> any files to download from it.

I didn't differentiate between spidering or direct links to external files. I simply iterated over all files that the pip PackageFinder was able to find, figured out the version for each url, and stored if that version came a link to a pypi.python.org resource or a different domain. I then diffed the two lists to get a list of versions that are _only_ installable externally. That 90% is 90% who can have *all* links what so ever besides ones hosted on PyPI itself removed and not have any versions be no longer installable.

> 
> I am saying that if a project has no *downloadable* files (not web
> pages) whose links can only be found by spidering, then we can turn
> off the rel attribute.
> 
> How many projects do not have any download links listed on their
> rel=""-linked pages?
> 
> 
>>> 1. Add a project-level option to enable or disable the adding of the
>>> rel="" attribute to /simple links (but not affecting the links in any
>>> other way)
>>> 2. Default it to disabled for new projects, and
>>> 3. Set it to disabled *now* for the 90% of projects that *don't have
>>> external files*?
>> 
>> +1 except 1. should be to remove the links entirely from the /simple/
>> index, not to just remove the rel attribute.
> 
> -1, since sometimes download links are in fact *download links*.  So
> this design choice would unncessarily limit the number of projects for
> whom the option could be applied automatically and immediately.
> 
> That is, a project with a download link of "foobar.com/foobar-1.2.tgz"
> would no longer be usable if you removed the download link from the
> /simple index, but would remain usable if the rel attribute were
> removed.


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

_______________________________________________
Catalog-SIG mailing list
[email protected]
http://mail.python.org/mailman/listinfo/catalog-sig

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)

iQIcBAEBAgAGBQJRPnVwAAoJEG48vOkzctz64HwQAKnPlf8kyXLcqjvhzVe7fleW
pSVyTfaE799yO2Rv97PqutkSKR4Xzh0JtP55owgeUS3rdLYT1jpF2gBOg8/LlGsH
miRAl2VnqkGwce6BS2E4UHxf5+9ZPZHrFY6QHJt8KCIsQdQL9WNYXed5gGuiIt6g
Cbz33o7cSC5P3mRQfEZVUh849T0Je5yTMsMZm6vdhs0DZeN5qtqftY1IE7uGLvwW
TwUiAoMRig87F+dCvcl0vxfME0mUkDis7mbW4lm0omzPkkXgRGuTaHyX3i5pMwEF
l7C3KVCtY816Tnwb4Mon/W/H7CnmlGR/f9iq/9LXevgasb0+kCyzfKVdegf0oCoq
lRpO3fytcs1G1bN/UdYtE1zJlL/0OlHlKrbhIE2UY/dlKlyoK/smlEVjoKuG/q6V
NTtrxQhd0PsW6a/biT83oHIixuRFYvVHBXTvL8YF8Gsur1a+eSEJOp5H9HCxdj4T
7WIrQ92GJu+lEshciReHmRuZRrQIVnJXGnVIb5euBT3/qN5MpbDEtChO+Ccu5UEM
bYGdO/prIjeKMSKx84TmtPhtLvz16+SSKt5FS5O4OgRiF9dWD8pG44AzZc6cusku
AJvJPAVrlRlYYNwWqW2kL3hNukVZOv1ZDLQ/4ecicqgmlgP5oYqDpk3VqGRlsgxf
Jq9JApmnF1V+mNtKmVTj
=LAQF
-----END PGP SIGNATURE-----