Re: pip installation for pypy is slow

On 29/11/21 6:01 am, Hao Wang wrote:

> Dear pypy-devs,
>
> The pip installation of pypy (pypy -m pip install scipy, for example) 
> is slower than python-pip clean. Is there a way to fix this problem ?
>
> Bravo!
> Hao Wang
>

Thanks for giving PyPy a try. When pre-compiled binary wheels are not 
available, python must compile from source. This takes time. We have 
been making an effort to convince project maintainers to provide binary 
wheels for PyPy. SciPy has a complicated workflow and a skeleton support 
team. That together with the fact that currently PyPy + SciPy is much 
slower to run CI (small snippets of code run in tests are not PyPy's 
strong point, and the c-api emulation layer makes things worse) mean 
that SciPy is hesitant to add support for PyPy, see the PR where the CI 
runs were removed [0] in 2019.


Our long-term proposal to solve this problem is to get pybind11 (for c++ 
code), cython (for c code), and f2py (for fortran code) to generate 
wrappers using HPy [1] instead of the C-API, which will speed up testing 
with PyPy.  A second possibility is that the scientific python community 
will move away from its heavy dependency on the CPython C-API and more 
towards using higher-level protocols [4]. Both of these efforts will 
take many years unless heavily sponsored.


A more specific short-term solution for your installation problem is to 
use conda-forge's miniforge [3]. This will solve the installation 
problem, but PyPy will still be slow on scipy-heavy workflows. The 
conda-forge project supplies binary packages for pypy3.7, and is 
discussing pypy3.8 support [2]:


conda create -n pypy3.7 pypy

conda activate pypy3.7

conda install scipy


Matti


[0] https://github.com/scipy/scipy/pull/10085

[1] https://hpyproject.org/

[2] https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/2089

[3] https://github.com/conda-forge/miniforge

[4] https://labs.quansight.org/blog/2021/11/pydata-extensibility-vision/

_______________________________________________
pypy-dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pypy-dev