What is the meaning of x[...] below?
a = np.arange(6).reshape(2,3) for x in np.nditer(a, op_flags=['readwrite']): x[...] = 2 * x What is the meaning of x[...] below?
a = np.arange(6).reshape(2,3) for x in np.nditer(a, op_flags=['readwrite']): x[...] = 2 * x While the proposed duplicate What does the Python Ellipsis object do? answers the question in a general python context, its use in an nditer loop requires, I think, added information.
https://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#modifying-array-values
Regular assignment in Python simply changes a reference in the local or global variable dictionary instead of modifying an existing variable in place. This means that simply assigning to x will not place the value into the element of the array, but rather switch x from being an array element reference to being a reference to the value you assigned. To actually modify the element of the array, x should be indexed with the ellipsis.
That section includes your code example.
So in my words, the x[...] = ... modifies x in-place; x = ... would have broken the link to the nditer variable, and not changed it. It's like x[:] = ... but works with arrays of any dimension (including 0d). In this context x isn't just a number, it's an array.
Perhaps the closest thing to this nditer iteration, without nditer is:
In [667]: for i, x in np.ndenumerate(a): ...: print(i, x) ...: a[i] = 2 * x ...: (0, 0) 0 (0, 1) 1 ... (1, 2) 5 In [668]: a Out[668]: array([[ 0, 2, 4], [ 6, 8, 10]]) Notice that I had to index and modify a[i] directly. I could not have used, x = 2*x. In this iteration x is a scalar, and thus not mutable
In [669]: for i,x in np.ndenumerate(a): ...: x[...] = 2 * x ... TypeError: 'numpy.int32' object does not support item assignment But in the nditer case x is a 0d array, and mutable.
In [671]: for x in np.nditer(a, op_flags=['readwrite']): ...: print(x, type(x), x.shape) ...: x[...] = 2 * x ...: 0 <class 'numpy.ndarray'> () 4 <class 'numpy.ndarray'> () ... And because it is 0d, x[:] cannot be used instead of x[...]
----> 3 x[:] = 2 * x IndexError: too many indices for array A simpler array iteration might also give insight:
In [675]: for x in a: ...: print(x, x.shape) ...: x[:] = 2 * x ...: [ 0 8 16] (3,) [24 32 40] (3,) this iterates on the rows (1st dim) of a. x is then a 1d array, and can be modified with either x[:]=... or x[...]=....
And if I add the external_loop flag from the next section, x is now a 1d array, and x[:] = would work. But x[...] = still works and is more general. x[...] is used all the other nditer examples.
In [677]: for x in np.nditer(a, op_flags=['readwrite'], flags=['external_loop']): ...: print(x, type(x), x.shape) ...: x[...] = 2 * x [ 0 16 32 48 64 80] <class 'numpy.ndarray'> (6,) Compare this simple row iteration (on a 2d array):
In [675]: for x in a: ...: print(x, x.shape) ...: x[:] = 2 * x ...: [ 0 8 16] (3,) [24 32 40] (3,) this iterates on the rows (1st dim) of a. x is then a 1d array, and can be modified with either x[:] = ... or x[...] = ....
Read and experiment with this nditer page all the way through to the end. By itself, nditer is not that useful in python. It does not speed up iteration - not until you port your code to cython.np.ndindex is one of the few non-compiled numpy functions that uses nditer.
The ellipsis ... means as many : as needed.
For people who don't have time, here is a simple example:
In [64]: X = np.reshape(np.arange(9), (3,3)) In [67]: Y = np.reshape(np.arange(2*3*4), (2,3,4)) In [70]: X Out[70]: array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) In [71]: X[:,0] Out[71]: array([0, 3, 6]) In [72]: X[...,0] Out[72]: array([0, 3, 6]) In [73]: Y Out[73]: array([[[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], [[12, 13, 14, 15], [16, 17, 18, 19], [20, 21, 22, 23]]]) In [74]: Y[:,0] Out[74]: array([[ 0, 1, 2, 3], [12, 13, 14, 15]]) In [75]: Y[...,0] Out[75]: array([[ 0, 4, 8], [12, 16, 20]]) In [76]: X[0,...,0] Out[76]: array(0) In [77]: Y[0,...,0] Out[77]: array([0, 4, 8]) This makes it easy to manipulate only one dimension at a time.
One thing - You can have only one ellipsis in any given indexing expression, or your expression would be ambiguous about how many : should be put in each.
as many : as needed, but in Y[...,0], how many are needed? None, I would argue. Would as many : as *possible* be more correct?... expands to the number of : objects needed for the selection tuple to index all dimensions." (NumPy User Guide, with Ellipsis replaced by the corresponding literal for clarity). For a more practical example see this answer.I believe a very good parallel (that most people are maybe used to) is to think that way:
import numpy as np random_array = np.random.rand(2, 2, 2, 2) In such case, [:, :, :, 0] and [..., 0] are the same.
You can use to analyse only an specific dimension, say you have a batch of 50 128x128 RGB image (50, 3, 128, 128), if you want to slice a piece of it in every image at every color channel, you could either do image[:,:,50:70, 20:80] or image[...,50:70,20:80]
Just be aware that you can't use it more than once in the statement like [...,0,...] is invalid.
No need for long explanations :-) In Numpy, the ellipsis (3 dots) just means "the number of : required for the index to match the shape of the array".
: itself is a shortcut to represent all elements in the dimension considered.
For example for a shape (n, p, q, r, s), that is an array with 5 dimensions, the ellipsis can be used to replace, 0, 1, 2, 3, 4 or 5 consecutive :. These forms are equivalent:
a[:, :, :, 4, 2] and a[..., 4, 2]a[4, :, :, :, 2] and a[4, ..., 2]a[4, 2, :, :, :] and a[4, 2, ...]a[:, :, :, :, 4] and a[..., 4]a[:, :, :, :, :] and a[...]and any variant with slices like:
a[1:2, :, :, :, 2:6] and a[1:2, ..., 2:6]This can be used to simplify indexes returning a slice of an array, instead of using consecutive : to include the corresponding whole dimensions. This simplification hides the actual shape of the array, and can lead to slicing errors.
Indeed there can be only one ellipsis in the index, else how would Numpy know how to distribute the required :?