ENH: allow scipy.sparse matrices to preserve `np.memmap` input data

Hi all - this is my first post here.

I've been using csr and coo matrices for large graph algorithms. I've found that scipy will beat frameworks like pregel on almost any task with large memory machines >200GB+ for storing large adjacency matrices. To run in environments with memory constraints, using numpy.memmap arrays for COO: (row, col, data) or CSR: (indices, indptr, data) has helped to reduce the memory footprint (obviously at the cost of some speed).

However, getting `csr_matrix` and `coo_matrix` to accept `np.memmap` format involved quite a bit of hacky monkey patching, because currently the scipy constructors will convert everything to a base ndarray, effectively copying to memory from disk. 

Would there be a good reason not to support this in a future version natively? The changes required would be just some usage of the `subok=True` kwarg for `np.array` which would preserve the `np.memmap` class and using `np.asanyarray` instead of `np.asarray` in other places.

For example, in the `csr_matrix.__init__`:  https://github.com/scipy/scipy/blob/979102e73a8b9612ac3c97c438e40de26e46c9b7/scipy/sparse/_compressed.py#L69-L72
```python
                    self.indices = np.array(indices, copy=copy,
                                            dtype=idx_dtype, subok=True)
                    self.indptr = np.array(indptr, copy=copy, 
                                           dtype=idx_dtype, subok=True)
                    self.data = np.array(data, copy=copy, 
                                         dtype=dtype, subok=True)
```
keeps input data on disk.

When the user then performs an operation that creates a new matrix, for example matrix multiplication, it becomes more difficult to keep things to memmaps, because the user has to specify a filepath to keep the new matrix at. But I think a great first step would be to not cast the input data - this still leads to a decrease in memory usage, and the user can choose to memmap the output manually themselves.

I'd be happy to raise a PR for this if this has some support - looking forward to hearing thoughts!
_______________________________________________
SciPy-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/scipy-dev.python.org/
Member address: [email protected]