Re: Idea: perennial manylinux tag

Hi,

As the original author of auditwheel and co-author of PEP 513, I figure I
should probably chime in.

I suspect that *I* am one of the major reasons that the manylinux1 ->
manylinux2010 transition has been unreasonably drawn out, rather than any
particular design flaw in the versioning scheme (manylinux_{cardinal
number} vs. manylinux_{year} vs. manylinux_{glibc version}).

I wrote auditwheel while I was finishing up graduate school. For years, the
"test suite" was just a couple wheel files hosted on my school's
per-student cgi-bin/ directory (which recently stopped working, since I'm
no longer a student). The logging was just random print statements. It
worked (I think), but it wasn't particularly well designed, nor was it well
tested. And it was basically a one-person project. After I finished my
Ph.D., I got a full time job and mostly stopped contributing to open
source. I think a lot of the reason for the delay in the manylinux2010
transition was that nobody was present and accounted for to develop
auditwheel.

Now, the project is in a much better place. Elana Hashman is leading doing
an awesome job leading auditwheel, and its seems like there's a new
momentum for manylinux2010. With now-proper maintenance and testing for
auditwheel, I don't think it will be as hard jump to the next iteration of
manylinux (e.g. manylinux2014) as it was to jump from manylinux1 to
manylinux2010.

-Robert

On Fri, Nov 30, 2018 at 3:12 AM Nathaniel Smith <[email protected]> wrote:

> Hi all,
>
> The manylinux1 -> manylinux2010 transition has turned out to be very
> difficult. Timeline so far:
>
> March 2017: CentOS 5 went EOL
> April 2018: PEP 517 accepted
> May 2018: support for manylinux2010 lands in warehouse
> November 2018: support lands in auditwheel, and pip master
> December 2018: 21 months after CentOS 5 EOL, wwee still don't have an
> official build environment, or support in a pip release
>
> We'll get through this, but it's been super painful and maybe we can
> change things somehow so it will suck less next time.
>
> We don't have anything like this pain on Windows or macOS. We never have
> to update pip, warehouse, etc., after those OSes hit EOLs. Why not?
>
> On Windows, we have just two tags: "win32" and "win_amd64". These are
> defined to mean something like "this wheel will run on any recent-ish
> Windows system". So the meaning of the tag actually changes over time: it
> used to be that if a wheel said it ran on win32, then that meant it would
> work on winxp, but since winxp hit EOL people started uploading "win32"
> wheels that don't work on winxp, and that's worked fine.
>
> On macOS, the tags look like "macosx_10_9_x86_64". So here we have the OS
> version embedded in the tag. This means that we do occasionally switch
> which tags we're using, kind of like how manylinux1 -> manylinux2010 is
> intended to work. But, unlike for the manylinux tags, defining a new macosx
> tag is totally trivial: every time a new OS version is released, the tag
> springs into existence without any human intervention. Warehouse already
> accepts uploads with this tag; pip already knows which systems can install
> wheels with this tag, etc.
>
> Can we take any inspiration from this for manylinux?
>
> We could do the Windows thing, and have a plain "manylinux" tag that means
> "any recent-ish glibc-based Linux". Today it would be defined to be "any
> distro newer than CentOS 6". When CentOS 6 goes out of service, we could
> tweak the definition to be "any distro newer than CentOS 7". Most parts of
> the toolchain wouldn't need to be updated, though, because the tag wouldn't
> change, and by assumption, enforcement wouldn't really be needed, because
> the only people who could break would be ones running on unsupported
> platforms. Just like happens on Windows.
>
> We could do the macOS thing, and have a "manylinux_${glibc version}" tag
> that means "this package works on any Linux using glibc newer than ${glibc
> version}". We're already using this as our heuristic to handle the current
> manylinux profiles, so e.g. manylinux1 is effectively equivalent to
> manylinux_2_5, and manylinux2010 will be equivalent to manylinux_2_12. That
> way we'd define the manylinux tags once, get support into pip and warehouse
> and auditwheel once, and then in the future the only thing that would have
> to change to support new distro releases or new architectures would be to
> set up a proper build environment.
>
> What do y'all think?
>
> -n
> --
> Distutils-SIG mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/6AFS4HKX6PVAS76EQNI7JNTGZZRHQ6SQ/
>


-- 
-Robert

--
Distutils-SIG mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at https://mail.python.org/archives/list/[email protected]/message/MUV6QHFHHY3OXVWCGL5RE5MYD7HD4JO2/