PyQt5 install fails with parallelized make

Dmitry Shachnev <[email protected]> Mon, 6 Jan 2020 19:23:26 +0300

Newsgroups	gmane.comp.python.pyqt-pykde
Message-ID	<[email protected]>

Hi Phil!

Currently the top-level Makefile generated by qmake has the following line:

  install:install_subtargets install_init_py install_uic_package install_pylupdate install_pyrcc install_tools install_pep484_stubs install_distinfo  FORCE

With this code, make may attempt to run all dependencies of install target
simultaneously.

However, install_distinfo will fail if it is run before install_subtargets
is complete.

In my case, the Debian build tools ran “make -j4 install” and it failed with
this error. See how QtQml.so is installed just after mk_distinfo.py fails:

  Traceback (most recent call last):
    File "/home/mitya57/pyqt5/mk_distinfo.py", line 112, in <module>
      fn_f = open(fn, 'rb')
  FileNotFoundError: [Errno 2] No such file or directory: '/home/mitya57/pyqt5/debian/tmp/usr/lib/python3.8/dist-packages/PyQt5/QtQml.so'
  make[2]: *** [Makefile:2104: install_distinfo] Error 1
  make[2]: *** Waiting for unfinished jobs....
  make[3]: Entering directory '/home/mitya57/pyqt5/build-3.8/QtQml'
  /usr/lib/qt5/bin/qmake -install qinstall /home/mitya57/pyqt5/build-3.8/QtQml/QtQml.so /home/mitya57/pyqt5/debian/tmp/usr/lib/python3.8/dist-packages/PyQt5/QtQml.so
  make[3]: Leaving directory '/home/mitya57/pyqt5/build-3.8/QtQml'

I am attaching a patch that fixes this. Not sure if it is an elegant solution,
but forcing -j1 seemed even less elegant to me.

Or maybe mk_distinfo.py should open files from build directory, not from the
directory they will be installed into?

--
Dmitry Shachnev

_______________________________________________
PyQt mailing list    [email protected]
https://www.riverbankcomputing.com/mailman/listinfo/pyqt

distinfo_dependencies.diff (text/x-diff, 2.7 KB)

From: Dmitry Shachnev <[email protected]>
Date: Mon, 6 Jan 2020 18:03:23 +0300
Subject: Make the install_distinfo target depend on all other targets

Otherwise when building in parallel, mk_distinfo.py will fail because
some files are not yet installed.
---
 configure.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/configure.py b/configure.py
index cf56cf3..24b5e15 100644
--- a/configure.py
+++ b/configure.py
@@ -1685,6 +1685,7 @@ INSTALLS += init_py
 ''' % (' '.join(subdirs), root_dir))
 
     all_installs.append(root_dir + '/__init__.py')
+    distinfo_dependencies = ['install_subtargets', 'install_init_py']
 
     # Install the uic module.
     out_f.write('''
@@ -1694,6 +1695,7 @@ INSTALLS += uic_package
 ''' % (source_path('pyuic', 'uic'), root_dir))
 
     all_installs.append(root_dir + '/uic')
+    distinfo_dependencies.append('install_uic_package')
 
     # Install the tool main scripts and wrappers.
     if wrappers:
@@ -1709,10 +1711,12 @@ INSTALLS += %s
 ''' % (tool, source_path('sip', tool, tool_main), tool, root_dir, tool))
 
                 all_installs.append(root_dir + '/' + tool_main)
+                distinfo_dependencies.append('install_' + tool)
 
             wrapper_exes.append(wrapper)
             all_installs.append(target_config.pyqt_bin_dir + '/' + wrapper)
 
+        distinfo_dependencies.append('install_tools')
         out_f.write('''
 tools.files = %s
 tools.path = %s
@@ -1739,6 +1743,7 @@ INSTALLS += sip%s
 ))
 
                     all_installs.append(mdir)
+                    distinfo_dependencies.append('install_sip%s' % mname)
 
     # Install the stub files.
     if target_config.py_version >= 0x030500 and target_config.pyqt_stubs_dir:
@@ -1755,6 +1760,7 @@ INSTALLS += pep484_stubs
         all_installs.extend(
                 [target_config.pyqt_stubs_dir + '/' + pyi
                         for pyi in pyi_names])
+        distinfo_dependencies.append('install_pep484_stubs')
 
     # Install the QScintilla .api file.
     if target_config.qsci_api:
@@ -1767,6 +1773,7 @@ INSTALLS += qscintilla_api
 ''' % qmake_quote(api_dir))
 
         all_installs.append(api_dir + '/PyQt5.api')
+        distinfo_dependencies.append('install_qscintilla_api')
 
     if distinfo:
         # The command to run to generate the .dist-info directory.
@@ -1779,8 +1786,9 @@ INSTALLS += qscintilla_api
         out_f.write('''
 distinfo.extra = %s
 distinfo.path = %s
+distinfo.depends = %s
 INSTALLS += distinfo
-''' % (run_mk_distinfo, root_dir))
+''' % (run_mk_distinfo, root_dir, ' '.join(distinfo_dependencies)))
 
         # Create the file containing all installed files.
         installed = open('installed.txt', 'w')

signature.asc (application/pgp-signature, 833 B)