Re: Parallel sparse matrix (CSR) operations?

I agree with Ralf: enabling parallelism for sparse operations would be
great, but we need to be careful to give users control over where and how
it's used.

Feel free to open a draft PR so we can discuss specifics.

On Wed, Dec 6, 2023, 12:47 PM Ralf Gommers <[email protected]> wrote:

>
>
> On Mon, Dec 4, 2023 at 8:59 AM <[email protected]> wrote:
>
>> Is there some appetite for parallelizing the basic sparse matrix
>> operations?
>>
>> The existing CSR methods for matrix construction, multiply, matvec,
>> binop, etc. are written in sequential C++ and offer some low-hanging fruit
>> for parallelism. The main loop for most of these is (or can easily be made)
>> embarrassingly parallel. As a proof of concept I converted the outer loops
>> of many methods to parallel STL algorithms and saw a 3-5x speedup on my
>> 6-core laptop (on medium/large versions of existing benchmarks).
>>
>> Operations on small matrices should remain sequential to avoid paying
>> parallel overhead when it won't be amortized.
>>
>> The parallel STL algorithms I used are implemented with a thread pool.
>> This avoids the TBB requirement for GCC's parallel STL and will work on all
>> platforms. I avoided OpenMP as per docs. If there is a different preferred
>> way to write parallel C++ in the SciPy codebase, let me know.
>>
>> If folks are interested I'd be happy to contribute a workable PR. The
>> code is cleaner than I expected it to be.
>>
>
> Hi Adam, thanks for experimenting with this. I think that yes, there is
> interest in adding parallel execution support to many APIs where that makes
> sense. It's basically the first item on the roadmap:
> http://scipy.github.io/devdocs/dev/roadmap.html#performance-improvements.
> The disclaimer there is that we can't just convert serial operations to
> parallel ones, because that makes composition with other libraries and end
> user code using multiprocessing et al. quite difficult (xref
> https://thomasjpfan.github.io/parallelism-python-libraries-design/ and
> https://www.youtube.com/watch?v=3h6cAzpWNVU). Instead, we need to use the
> `workers=` keyword pattern to let users and downstream library authors opt
> in to parallel execution. You can look at, e.g., `fft` and `optimize`
> functions that have this keyword. `fft` functions use pthreads, so is
> probably the best example.
>
> Cheers,
> Ralf
>
> _______________________________________________
> SciPy-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> Member address: [email protected]
>

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]