How do dunder methods getitem and len provide iteration? [duplicate]

Question

I am reading about Python's dunder methods. One of the things I learned is that if a class provides an implementation for __getitem__ and __len__, it can be used in a for loop.

Looking at the built-in classes like list, tuple, and range I noticed that all of them provide an implementation for __iter__ which returns an iterator for a corresponding type. My understanding is that for loop uses this iterator to traverse the elements.

However, how does it work for a class which provides __getitem__ and __len__ but not an __iter__?

As an example, here's a Range class which mimics the in-built range:

class Range(): def __init__(self, start, stop=None, step=1): if step == 0: raise ValueError('step cannot be 0') if stop is None: start, stop = 0, start self._length = max(0, (stop - start + step - 1) // step) self._start = start self._step = step def __len__(self): return self._length def __getitem__(self, k): if k < 0: k = len(self) + k if not 0 <= k < self._length: raise IndexError('Index out of range') return self._start + (k * self._step)

Iterating over it with a for loop:

In [21]: for elem in Range(5): ...: print(elem) ...: 0 1 2 3 4

Jonathan1609 · Accepted Answer · 2023-02-27 22:22:03Z

A iterable is a class which defines either __iter__ or __getitem__, no need for __len__.

The difference between the __iter__ implementation and the __getitem__ implementation is: __iter__ calls __next__ on the object that returned from __iter__ (aka iterator), until it reaches StopIteration and that's where the for loop stops. However __getitem__, starts from zero (always), and each iteration it increments by one, until it reaches IndexError, and it does that by obj[idx].

For instance:

class GetItem: def __getitem__(self, idx): if idx == 10: raise IndexError return idx for i in GetItem(): print(i)

The result will be

0 1 2 ... 9

because as soon as the index gets to 10, it raises IndexError and the loop stops.

__iter__ on the other hand,

class Iter: def __iter__(self): self.n = 0 return self def __next__(self): self.n += 1 if self.n == 10: raise StopIteration return self.n for i in Iter(): print(i)

Here, you need to keep track of the state by yourself, whereas in __getitem__ it does it by itself, it's better for counting/indexing and such.

"However getitem, starts from zero (always)" Is that really the case? In dictionaries, doesn't __getitem__ take the dictionary's keys?
@krm There is a precedence for __iter__ over __getitem__ (the __getitem__ will take the control over the iteration just if there is no __iter__ defined). In dictionaries, there is __iter__ defined which returns an iterator over the keys of the dict, and the __getitem__ returns the value for a particular key, as expected.

chepner · Accepted Answer · 2021-07-04 13:22:18Z

You can implement for x in y: ... in two ways.

Rewrite as an infinite while loop that calls next explicitly.

itr = iter(y) # Using __iter__ while True: try: x = next(itr) except StopIteration: break ...

Rewrite as an iteration over range(len(y)):
```
for i in range(len(y)): # Using __len__ x = y[i] # Using __getitem__ ... 
```
This relies on __getitem__ being defined for indices 0 through len(y) - 1.

Update: as @Jonathon1609 reminds me, __len__ is not used. Instead, the for loop requires __getitem__ to raise an IndexError to terminate iteration.

i = 0 while True: try: x = y[i] # Uses __getitem__ except IndexError: break ... i += 1

reversed is the function that can use __len__ and __getitem__ together if __reversed__ is not defined.

this is helpful. Is this how for loop works internally, too? What's going on under the hood with the custom Range class?
@AnSOUser Under the hood, a for loop is (mostly) equivalent to the while loop, though with some modifications to handle the else clause (and it doesn't create a visible variable itr in the current scope).

Collectives™ on Stack Overflow

How do dunder methods getitem and len provide iteration? [duplicate]

2 Answers 2

2 Comments

3 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Linked

Related