12

First I want to clarify, I'm NOT asking what is "iterator".

This is how the term "iterable" is defined in Python's doc:

iterable

An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() or __getitem__() method.

Iterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), ...). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop.

See also iterator, sequence, and generator.

As other people suggested, using isinstance(e, collections.Iterable) is the most pythonic way to check if an object is iterable.
So I did some test with Python 3.4.3:

from collections.abc import Iterable class MyTrain: def __getitem__(self, index): if index > 3: raise IndexError("that's enough!") return index for name in MyTrain(): print(name) # 0, 1, 2, 3 print(isinstance(MyTrain(), Iterable)) # False

The result is quite strange: MyTrain has defined __getitem__ method, but it is not considered as an iterable object, not to mention it's capable of returning one number at a time.

Then I removed __getitem__ and added the __iter__ method:

from collections.abc import Iterable class MyTrain: def __iter__(self): print("__iter__ called") pass print(isinstance(MyTrain(), Iterable)) # True for name in MyTrain(): print(name) # TypeError: iter() returned non-iterator of type 'NoneType'

It is now considered as a "true" iterable object in spite of it cannot produce anything while iterating.

So did I misunderstand something or is the documentation incorrect?

Improve this question
16
  • 8
    isinstance won't check that the interface is implemented correctly, that doesn't get found out until you actually try to iterate over it, just that the appropriate method(s) (in this case only __iter__) are available. Commented Sep 26, 2015 at 17:33
  • 1
    docs.python.org/3/library/… Commented Sep 26, 2015 at 17:35
  • 15
    "using isinstance(e, collections.Iterable) is the most pythonic way to check if an object is iterable" - no, I would say that trying to iterate over it is the most Pythonic way! Commented Sep 26, 2015 at 17:36
  • 2
    to test if something is an iterable, I do a try / except block where I try var = iter(var), if it throws and exception, then it's not an iterable Commented Sep 26, 2015 at 17:58
  • 3
    @jonrsharpe. But that might be self-defeating, since it could consume the iterable. Maybe a better check would be to try iter(x) and see if it raises a TypeError. Commented Sep 26, 2015 at 18:03

3 Answers 3

Reset to default
9

I think the point of confusion here is that, although implementing __getitem__ does allow you to iterate over an object, it isn't part of the interface defined by Iterable.

The abstract base classes allow a form of virtual subclassing, where classes that implement the specified methods (in the case of Iterable, only __iter__) are considered by isinstance and issubclass to be subclasses of the ABCs even if they don't explicitly inherit from them. It doesn't check whether the method implementation actually works, though, just whether or not it's provided.

For more information, see PEP-3119, which introduced ABCs.

using isinstance(e, collections.Iterable) is the most pythonic way to check if an object is iterable

I disagree; I would use duck-typing and just attempt to iterate over the object. If the object isn't iterable a TypeError will be raised, which you can catch in your function if you want to deal with non-iterable inputs, or allow to percolate up to the caller if not. This completely side-steps how the object has decided to implement iteration, and just finds out whether or not it does at the most appropriate time.

As the ABC documentation mentions:

Checking isinstance(obj, Iterable) detects classes that are registered as Iterable or that have an __iter__() method, but it does not detect classes that iterate with the __getitem__() method. The only reliable way to determine whether an object is iterable is to call iter(obj).

To add a little more, I think the docs you've quoted are slightly misleading. To quote the iter docs, which perhaps clear this up:

object must be a collection object which supports the iteration protocol (the __iter__() method), or it must support the sequence protocol (the __getitem__() method with integer arguments starting at 0).

This makes it clear that, although both protocols make the object iterable, only one is the actual "iteration protocol", and it is this that isinstance(thing, Iterable) tests for. Therefore we could conclude that one way to check for "things you can iterate over" in the most general case would be:

isinstance(thing, (Iterable, Sequence))

However, this doesn't work because:

  1. your class doesn't implement __len__ (and doesn't need to anyway, being iterable only requires a sequence-compatible __getitem__); and

  2. Sequence is a complex interface:

    Complex interfaces do not support [virtual subclassing] because an interface is more than just the presence of method names. Interfaces specify semantics and relationships between methods that cannot be inferred solely from the presence of specific method names.

    so even a class that implements both __getitem__ and __len__ doesn't automatically become a Sequence, it needs to explicitly extend the ABC (at which point it would inherit an __iter__ implementation and become Iterable anyway).

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Thx jorsharpe, your answer is of great help. For now, I really think the doc I quoted is misleading. The glossary "iterable" in the glossary doc means "can be iterated" no matter how it's done, while Python's collections.abc.Iterable represents valid iteration container(I made this word up), that is whatever object that defines the __iter__ method.
So collections.abc.Iterable is not necessarily "iterable" since it could return an invalid iterator which does not implement __next__. Meanwhile , an "iterable" object could have no relation with collections.abc.Iterable by only defining __getitem__. What do you think?
@laike9m that's right, but then simply defining a __getitem__ method is also no guarantee that it actually works! This is why I prefer the duck typing "EAFP" approach - given that you can't truly know if an object is actually iterable until you do it, then that's the most appropriate time to deal with it not being. Ideally the documentation should say something like "an object is iterable if it implements the iterator or sequence protocols", perhaps, although it seems that __len__ isn't needed in practice.
Every collections.abc.Sequence object is a special case of collections.abc.Iterable (because every Sequence object has __iter__ method). So the isinstance(thing, (Iterable, Sequence)) test doesn't make sense for me - it doesn't have any advantage over isinstance(thing, Iterable).
@Rodvi I thought if a class implemented __getitem__ and __len__ it would virtually subclass Sequence, hence the final sentence of my answer, but apparently this is a "complex interface" that doesn't do that. Updated.
|
0

It is an iterable. However you haven't inherited from abc.Iterable, so naturally Python won't report it as being descended from that class. The two things -being an iterable, and descending from that base class - are quite separate.

Improve this answer

1 Comment

I think what's confused the OP is that classes implementing __iter__ are virtual subclasses of Iterable even if they don't explicitly inherit from it.
-1

Iterable is something(collection anything) that allows some kind of iteration on its elements. But what is the generic way of iteration in python? That is using - in keyword, which uses __iter__ method of an object. So, in that terms any object that defines __iter__ can be used with in, and is an Iterable.

So, most 'duck-typish' way to check if an object is iterable is if an object is this, (Yeah, I know implicitly that is what's happening in isinstance case as well, due to virtual classes)

hasattr(train, '__iter__')

because according to duck-typing, we care about behavior provided by an object instead of its ancestory.

If you have a faulty implementation of __iter__ that doesn't means object is not iterable, it just means that you have bug in your code.

Note:- Those objects that don't define __iter__ can still be iterable in general sense, by using some other method, it's just they can't be used with in keyword. E.g.:- NumberList instance is iterable over each method, but isn't iterable in python sense.

class NumberList: def __init__(self, values): self.values = values def each(self): return self.values
Improve this answer

7 Comments

1. That's not duck typing. 2. hasattr(train, '__iter__') is roughly what isinstance(train, Iterable) already does.
@jonrshare I know, but that is bit misleading. Don't you think isinstance should check inheritance instead
If an object provides 'iter' method, we don't care whether it's instance of Iterable or not. Isn't that what duck typing means
"Don't you think isinstance should check inheritance instead" - no, I think it should do what it's documented to: "Return true if the object argument is an instance of the classinfo argument, or of a (direct, indirect or virtual) subclass thereof." (where for virtual see docs.python.org/3/glossary.html#term-abstract-base-class)
Duck typing means not using hasattr at all, just writing for thing in train: and either dealing with the error in the function or letting the caller catch it.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.