Proposal for timestamps of installed files

Proposal: change timestamps of files installed from a wheel to the time 
of installation, except for .py files.

**Existing situation**

Let's consider two ways of installing a Python package: installation 
from a source tarball and installation from a wheel, in both cases using 
"pip install".

The installation roughly works as follows:

(A) Create a temporary directory for pip to work in, with a filename 
like /tmp/pip-6N_2Kx-build

(B) Extract the archive (sources or wheel) somewhere in the temporary 
directory

(C) Build the package in the temporary directory (in both cases, this 
involves creating .pyc files for the .py files)

(D) Copy the installation from the temporary directory to the final 
installation directory (typically somewhere in site-packages)

(E) Delete the temporary directory

Both (B) and (D) involve a copy operation of some sort, so the question 
is: what to do with timestamps? In both cases, (D) currently preserves 
the timestamp of the files. For (B), timestamps are preserved when 
extracting a tarball but not when extracting a wheel.

**Proposal**

Change step (D) for wheel installs to NOT preserve timestamps. It might 
be safer to make sure that all installed files have the same timestamp, 
so this would mean: give all installed files the timestamp of a fixed 
time, namely the time when step (D) was started in the installation process.

Important exception: for .py files, do preserve the timestamp.

I think that (B) should preserve timestamps. However, that is a 
different issue which should be fixed independently.

**The problem**

For a source installation, the timestamps are preserved in both (B) and 
(D), so the timestamp of the installed files can be much older than the 
time of installation.

This is a problem for dependency checking: suppose that I am the 
developer of a package MYPKG which has a dependency on the C API of a 
package OTHERPKG. The C API of OTHERPKG is defined by .h files which are 
installed by OTHERPKG. The package MYPKG correctly declares this 
dependency using the "depends" keyword of the Extension objects. Now, 
when I install a different version of OTHERPKG, I want that distutils 
notices that the dependency has been changed and automatically rebuilds 
the Extensions with that dependency.

This requires that the timestamp of the installed .h files is more 
recent than the timestamp of the compiled extension. In other words, the 
timestamp of installed files should be the time of installation.

A similar situation happens for a Cython API exposed by installed .pxd 
(and possibly also .h) files.

**Comparison with automake**

Automake (which is probably the most widely used build system for open 
source/free software) installs files with timestamp the time of 
installation. So, my proposal is doing what automake already does.

**The strange case of .pyc files**

As noted in https://github.com/pypa/pip/issues/5648 it is important that 
the timestamp of .py files are preserved after the .pyc file is created. 
This is because Python checks the exact timestamp of the .py file to 
determine whether it is up to date.

This explains the exception for .py files in the proposal.

**Backwards compatibility**

In other words: won't this break packages which rely on timestamps to be 
preserved?

When installing a wheel, timestamps from the wheel are already not 
preserved in step (B). So any package which would rely on preserving 
timestamps would already be broken when installed as wheel. So this 
proposal can only break stuff which depends on timestamps from step (C), 
which happens to be the case for .py files.

In my proposal, I'm only suggesting to make changes for wheel installs. 
I do think that the change makes sense for installs from source, but 
that will be irrelevant after PEP 517 is implemented anyway.


Jeroen.
--
Distutils-SIG mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at https://mail.python.org/mm3/archives/list/[email protected]/message/SOBBI2YA7TVEUI3EVXJX2Z4EDJOU4WYY/