Ad
Is There A Memory-efficient Way Of Inserting Zero-columns And Rows Into A Numpy Array?
I have a big (symmetric) numpy matrix arr
. It is of shape arr.shape = (50_000, 50_000)
. I want to insert some zero-rows/columns (in a symmetric way). Let's say the amount of rows / columns I want to insert is maybe 123
.
Small Example
import numpy as np
# Create symmetric square matrix
size = 3
arr = np.array(list(range(size**2))).reshape(size, size)
arr = arr + arr.T
# insert zero columns
# Each "1" represents a column from the original matrix, e.g.
# the first 1 is the first column of arr, the second 1 the second column of arr
# and so on
insert_cols = [1, 0, 0, 1, 0, 1, 0, 0]
# insert the zero rows / columns
current_index = 0
for col in insert_cols:
if col == 0:
arr = np.insert(arr, current_index, 0, axis=0)
arr = np.insert(arr, current_index, 0, axis=1)
current_index += 1
print(arr)
If I understand np.insert
correctly, then this code creates a copy of the array and copies the content all the time.
Question
I thought maybe this might be simpler / more efficient with one of the sparse matrix classes? Is there another way to make this more efficient?
Ad
Answer
Given insert_cols
, we can do something like this -
n = len(insert_cols)
out = np.zeros((n,n),arr.dtype)
idx = np.flatnonzero(insert_cols)
out[np.ix_(idx,idx)] = arr # or out[idx[:,None],idx] = arr
Alternatively, use a boolean version for indexing. Hence -
insert_cols_bool = np.asarray(insert_cols, dtype=bool)
Then, use insert_cols_bool
in place of idx
.
Sparse-matrix
To be more memory-efficient, we can store the output as a sparse matrix -
from scipy.sparse import coo_matrix
l = len(idx)
r,c = np.broadcast_to(idx[:,None],(l,l)).ravel(),np.broadcast_to(idx,(l,l)).ravel()
out = coo_matrix((arr.ravel(), (r,c)), shape=(n,n))
Ad
source: stackoverflow.com
Related Questions
- → What are the pluses/minuses of different ways to configure GPIOs on the Beaglebone Black?
- → Django, code inside <script> tag doesn't work in a template
- → React - Django webpack config with dynamic 'output'
- → GAE Python app - Does URL matter for SEO?
- → Put a Rendered Django Template in Json along with some other items
- → session disappears when request is sent from fetch
- → Python Shopify API output formatted datetime string in django template
- → Can't turn off Javascript using Selenium
- → WebDriver click() vs JavaScript click()
- → Shopify app: adding a new shipping address via webhook
- → Shopify + Python library: how to create new shipping address
- → shopify python api: how do add new assets to published theme?
- → Access 'HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT' with Python Shopify Module
Ad