in

Assigning maxtrix from scipy.sparse.identity to csr_matrix


I want to assign a large scale scipy.sparse.identity to a slice of scipy.sparse.csr_matrix but am failing to do so. In this case, m = 25000000 and p=3. Tc_temp is the csr_matrix of size 25000000 x 75000000.

Tc_temp = csr_matrix((m, p * m))
Tc_temp[0: m, np.arange(j, p * m + j, p)] = identity(m, format="csr")

The error traceback I get is:

Traceback (most recent call last):
  File "C:Program FilesJetBrainsPyCharm Community Edition 2021.2pluginspython-cehelperspydev_pydevd_bundlepydevd_exec2.py", line 3, in Exec
    exec(exp, global_vars, local_vars)
  File "<input>", line 1, in <module>
  File "C:UserskusariMiniconda3envscvxpy_envlibsite-packagesscipysparse_index.py", line 116, in __setitem__
    self._set_arrayXarray_sparse(i, j, x)
  File "C:UserskusariMiniconda3envscvxpy_envlibsite-packagesscipysparsecompressed.py", line 816, in _set_arrayXarray_sparse
    self._zero_many(*self._swap((row, col)))
  File "C:UserskusariMiniconda3envscvxpy_envlibsite-packagesscipysparsecompressed.py", line 932, in _zero_many
    i, j, M, N = self._prepare_indices(i, j)
  File "C:UserskusariMiniconda3envscvxpy_envlibsite-packagesscipysparsecompressed.py", line 882, in _prepare_indices
    i = np.array(i, dtype=self.indices.dtype, copy=False, ndmin=1).ravel()
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 233. GiB for an array with shape (62500000000,) and data type int32

The sparse.identity is somehow getting converted to dense matrix.



Source: https://stackoverflow.com/questions/70633388/assigning-maxtrix-from-scipy-sparse-identity-to-csr-matrix

a classic doom and wolfenstein3D like game made in Python, using the famous PYGAME module

Titan EyeX Smart Glasses Launched in India at a Price of Rs 9,999