Re: Proposal for timestamps of installed files

Does this proposal mean that .pyc files would need to be regenerated after
installing from a wheel (because the timestamp inside the .pyc file would
no longer match the .py file from which it came)? I could be wrong, but it
seems like it’s a desirable feature that .pyc files be generated and up to
date after installation.

I notice in your problem statement, you only describe that it is a problem
for .h and .pxd files not to get new timestamps.

—Chris

On Mon, Aug 6, 2018 at 3:55 AM Jeroen Demeyer <[email protected]> wrote:

> Proposal: change timestamps of files installed from a wheel to the time
> of installation, except for .py files.
>
> **Existing situation**
>
> Let's consider two ways of installing a Python package: installation
> from a source tarball and installation from a wheel, in both cases using
> "pip install".
>
> The installation roughly works as follows:
>
> (A) Create a temporary directory for pip to work in, with a filename
> like /tmp/pip-6N_2Kx-build
>
> (B) Extract the archive (sources or wheel) somewhere in the temporary
> directory
>
> (C) Build the package in the temporary directory (in both cases, this
> involves creating .pyc files for the .py files)
>
> (D) Copy the installation from the temporary directory to the final
> installation directory (typically somewhere in site-packages)
>
> (E) Delete the temporary directory
>
> Both (B) and (D) involve a copy operation of some sort, so the question
> is: what to do with timestamps? In both cases, (D) currently preserves
> the timestamp of the files. For (B), timestamps are preserved when
> extracting a tarball but not when extracting a wheel.
>
> **Proposal**
>
> Change step (D) for wheel installs to NOT preserve timestamps. It might
> be safer to make sure that all installed files have the same timestamp,
> so this would mean: give all installed files the timestamp of a fixed
> time, namely the time when step (D) was started in the installation
> process.
>
> Important exception: for .py files, do preserve the timestamp.
>
> I think that (B) should preserve timestamps. However, that is a
> different issue which should be fixed independently.
>
> **The problem**
>
> For a source installation, the timestamps are preserved in both (B) and
> (D), so the timestamp of the installed files can be much older than the
> time of installation.
>
> This is a problem for dependency checking: suppose that I am the
> developer of a package MYPKG which has a dependency on the C API of a
> package OTHERPKG. The C API of OTHERPKG is defined by .h files which are
> installed by OTHERPKG. The package MYPKG correctly declares this
> dependency using the "depends" keyword of the Extension objects. Now,
> when I install a different version of OTHERPKG, I want that distutils
> notices that the dependency has been changed and automatically rebuilds
> the Extensions with that dependency.
>
> This requires that the timestamp of the installed .h files is more
> recent than the timestamp of the compiled extension. In other words, the
> timestamp of installed files should be the time of installation.
>
> A similar situation happens for a Cython API exposed by installed .pxd
> (and possibly also .h) files.
>
> **Comparison with automake**
>
> Automake (which is probably the most widely used build system for open
> source/free software) installs files with timestamp the time of
> installation. So, my proposal is doing what automake already does.
>
> **The strange case of .pyc files**
>
> As noted in https://github.com/pypa/pip/issues/5648 it is important that
> the timestamp of .py files are preserved after the .pyc file is created.
> This is because Python checks the exact timestamp of the .py file to
> determine whether it is up to date.
>
> This explains the exception for .py files in the proposal.
>
> **Backwards compatibility**
>
> In other words: won't this break packages which rely on timestamps to be
> preserved?
>
> When installing a wheel, timestamps from the wheel are already not
> preserved in step (B). So any package which would rely on preserving
> timestamps would already be broken when installed as wheel. So this
> proposal can only break stuff which depends on timestamps from step (C),
> which happens to be the case for .py files.
>
> In my proposal, I'm only suggesting to make changes for wheel installs.
> I do think that the change makes sense for installs from source, but
> that will be irrelevant after PEP 517 is implemented anyway.
>
>
> Jeroen.
> --
> Distutils-SIG mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at
> https://mail.python.org/mm3/archives/list/[email protected]/message/SOBBI2YA7TVEUI3EVXJX2Z4EDJOU4WYY/
>

--
Distutils-SIG mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at https://mail.python.org/mm3/archives/list/[email protected]/message/FNNHIJX7PEW4P7LSBWRQRDJ3HM2JLSPC/