Re: Parallel sparse matrix (CSR) operations?

As a heavy user of parallel sparse operations, I would be DELIGHTED to see
a native SciPy implementation. Currently, I use PyRSB and/or the beta
functionality available through PyTorch. Both work well and greatly
accelerate computations. Looking forward to it and I would be happy to test
the prototype PR Adam,

Jorge

On Wed, Dec 6, 2023 at 10:56 AM CJ Carey <[email protected]> wrote:

> I agree with Ralf: enabling parallelism for sparse operations would be
> great, but we need to be careful to give users control over where and how
> it's used.
>
> Feel free to open a draft PR so we can discuss specifics.
>
> On Wed, Dec 6, 2023, 12:47 PM Ralf Gommers <[email protected]> wrote:
>
>>
>>
>> On Mon, Dec 4, 2023 at 8:59 AM <[email protected]> wrote:
>>
>>> Is there some appetite for parallelizing the basic sparse matrix
>>> operations?
>>>
>>> The existing CSR methods for matrix construction, multiply, matvec,
>>> binop, etc. are written in sequential C++ and offer some low-hanging fruit
>>> for parallelism. The main loop for most of these is (or can easily be made)
>>> embarrassingly parallel. As a proof of concept I converted the outer loops
>>> of many methods to parallel STL algorithms and saw a 3-5x speedup on my
>>> 6-core laptop (on medium/large versions of existing benchmarks).
>>>
>>> Operations on small matrices should remain sequential to avoid paying
>>> parallel overhead when it won't be amortized.
>>>
>>> The parallel STL algorithms I used are implemented with a thread pool.
>>> This avoids the TBB requirement for GCC's parallel STL and will work on all
>>> platforms. I avoided OpenMP as per docs. If there is a different preferred
>>> way to write parallel C++ in the SciPy codebase, let me know.
>>>
>>> If folks are interested I'd be happy to contribute a workable PR. The
>>> code is cleaner than I expected it to be.
>>>
>>
>> Hi Adam, thanks for experimenting with this. I think that yes, there is
>> interest in adding parallel execution support to many APIs where that makes
>> sense. It's basically the first item on the roadmap:
>> http://scipy.github.io/devdocs/dev/roadmap.html#performance-improvements.
>> The disclaimer there is that we can't just convert serial operations to
>> parallel ones, because that makes composition with other libraries and end
>> user code using multiprocessing et al. quite difficult (xref
>> https://thomasjpfan.github.io/parallelism-python-libraries-design/ and
>> https://www.youtube.com/watch?v=3h6cAzpWNVU). Instead, we need to use
>> the `workers=` keyword pattern to let users and downstream library authors
>> opt in to parallel execution. You can look at, e.g., `fft` and `optimize`
>> functions that have this keyword. `fft` functions use pthreads, so is
>> probably the best example.
>>
>> Cheers,
>> Ralf
>>
>> _______________________________________________
>> SciPy-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
>> Member address: [email protected]
>>
> _______________________________________________
> SciPy-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/scipy-dev.python.org/
> Member address: [email protected]
>


-- 
*Jorge E. Guerra, PhD.*
*Project Scientist, UCAR,AOML/NOAA*
*Earth Systems Research Laboratory, Boulder, CO*

_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]