Re: Parallel sparse matrix (CSR) operations?

On Mon, Dec 4, 2023 at 8:59 AM <[email protected]> wrote:

> Is there some appetite for parallelizing the basic sparse matrix
> operations?
>
> The existing CSR methods for matrix construction, multiply, matvec, binop,
> etc. are written in sequential C++ and offer some low-hanging fruit for
> parallelism. The main loop for most of these is (or can easily be made)
> embarrassingly parallel. As a proof of concept I converted the outer loops
> of many methods to parallel STL algorithms and saw a 3-5x speedup on my
> 6-core laptop (on medium/large versions of existing benchmarks).
>
> Operations on small matrices should remain sequential to avoid paying
> parallel overhead when it won't be amortized.
>
> The parallel STL algorithms I used are implemented with a thread pool.
> This avoids the TBB requirement for GCC's parallel STL and will work on all
> platforms. I avoided OpenMP as per docs. If there is a different preferred
> way to write parallel C++ in the SciPy codebase, let me know.
>
> If folks are interested I'd be happy to contribute a workable PR. The code
> is cleaner than I expected it to be.
>

Hi Adam, thanks for experimenting with this. I think that yes, there is
interest in adding parallel execution support to many APIs where that makes
sense. It's basically the first item on the roadmap:
http://scipy.github.io/devdocs/dev/roadmap.html#performance-improvements.
The disclaimer there is that we can't just convert serial operations to
parallel ones, because that makes composition with other libraries and end
user code using multiprocessing et al. quite difficult (xref
https://thomasjpfan.github.io/parallelism-python-libraries-design/ and
https://www.youtube.com/watch?v=3h6cAzpWNVU). Instead, we need to use the
`workers=` keyword pattern to let users and downstream library authors opt
in to parallel execution. You can look at, e.g., `fft` and `optimize`
functions that have this keyword. `fft` functions use pthreads, so is
probably the best example.

Cheers,
Ralf

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]