Is There A Way To Not Make A Copy When A Numpy Array Is Sliced?
I need to handle some large
numpy arrays in my project. After such an array is loaded from the disk, over half of my computer's memory will be consumed.
After the array is loaded, I make several slices (almost half of the array will be selected) of it, then I receive error tells me the memory is insufficient.
By doing a little experiment I understand, I receive the error because when a
numpy array is sliced, a copy will be created
import numpy as np tmp = np.linspace(1, 100, 100) inds = list(range(100)) tmp_slice = tmp[inds] assert id(tmp) == id(tmp_slice)
Is there a way that a slice of a
numpy array only refers to the memory addresses of the original array thus data entries are not copied?
slice is a well defined class, with
step values. It is used when we index a list with
alist[1: 10: 2]. This makes a new list with copies of the pointers from the original. In
numpy these are used in
basic indexing, e.g.
arr[:3, -3:]. This creates a
view of the original. The
view shares the data buffer, but has its own
But when we index arrays with lists, arrays or boolean arrays (mask), it has to make a copy, an array with its own data buffer. The selection of elements is too complex or irregular to express in terms of the
In some cases the index array is small (compared to the original) and copy is also small. But if we are permuting the whole array, then the index array, and copy will both be as large as the original.
- → What are the pluses/minuses of different ways to configure GPIOs on the Beaglebone Black?
- → Django, code inside <script> tag doesn't work in a template
- → React - Django webpack config with dynamic 'output'
- → GAE Python app - Does URL matter for SEO?
- → Put a Rendered Django Template in Json along with some other items
- → session disappears when request is sent from fetch
- → Python Shopify API output formatted datetime string in django template
- → Shopify app: adding a new shipping address via webhook
- → Shopify + Python library: how to create new shipping address
- → shopify python api: how do add new assets to published theme?
- → Access 'HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT' with Python Shopify Module