330

I have a list that contains many sub-lists of 3 elements each, like:

my_list = [["a", "b", 0], ["c", "d", 0], ["e", "f", 0], .....]

The last element of each sub-list is a sort of flag, which is initially 0 for each sub-list. As my algorithm progresses, I want to check whether this flag is 0 for at least one element. Currently I use a while loop, like so:

def check(list_): for item in list_: if item[2] == 0: return True return False

The overall algorithm loops as long as that condition is satisfied, and sets some of the flags in each iteration:

while check(my_list): for item in my_list: if condition: item[2] = 1 else: do_sth()

Because it causes problems to remove elements from the list while iterating over it, I use these flags to keep track of elements that have already been processed.

How can I simplify or speed up the code?

See also Pythonic way of checking if a condition holds for any element of a list for checking the condition for any element. Keep in mind that "any" and "all" checks are related through De Morgan's law, just as "or" and "and" are related.

Existing answers here use the built-in function all to do the iteration. See How do Python's any and all functions work? for an explanation of all and its counterpart, any.

If the condition you want to check is "is found in another container", see How to check if all of the following items are in a list? and its counterpart, How to check if one of the following items is in a list?. Using any and all will work, but more efficient solutions are possible.

Improve this question
3
  • 4
    Looks like your data structure is not ideal for your problem. If you explained the context a little more maybe we could suggest something more appropriate. Commented May 19, 2012 at 14:50
  • Maybe you could replace the items with None or [] as you iterate over the list instead of removing them. Checking the whole list with 'check()` iterating over all the items before each pass on the inner loop is a very slow approach. Commented May 19, 2012 at 20:08
  • I edited this question to try to make it easier to read and meet current style guidelines; but now it seems evident that it was originally motivated by an XY problem. It would have been better (and actually addressed the performance concern cited) to filter the list properly, rather than look for "better" ways to handle the flags. Commented Oct 17, 2023 at 17:27

5 Answers 5

Reset to default
568

The best answer here is to use all(), which is the builtin for this situation. We combine this with a generator expression to produce the result you want cleanly and efficiently. For example:

>>> items = [[1, 2, 0], [1, 2, 0], [1, 2, 0]] >>> all(flag == 0 for (_, _, flag) in items) True >>> items = [[1, 2, 0], [1, 2, 1], [1, 2, 0]] >>> all(flag == 0 for (_, _, flag) in items) False

Note that all(flag == 0 for (_, _, flag) in items) is directly equivalent to all(item[2] == 0 for item in items), it's just a little nicer to read in this case.

And, for the filter example, a list comprehension (of course, you could use a generator expression where appropriate):

>>> [x for x in items if x[2] == 0] [[1, 2, 0], [1, 2, 0]]

If you want to check at least one element is 0, the better option is to use any() which is more readable:

>>> any(flag == 0 for (_, _, flag) in items) True
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

My fault on the use of lambda, Python's all does not accept a function as the first argument like Haskell et. al., I changed my answer to a list comprehension as well. :)
@HampusNilsson A list comprehension is not the same as a generator expression. As all() and any() short circuit, if, for example, the first value on mine evaluates to False, all() will fail and not check any more values, returning False. Your example will do the same, except it will generate the entire list of comparisons first, meaning a lot of processing for nothing.
it also works if you do all([ cond(i) for i in range (n) ])
@CharlieParker Making it a list comprehension like that just makes things slower, as discussed in my previous comment. You want to use a generator expression (no square brackets) because that way it can short-circuit.
@GarethLatty ah, interesting point. Not sure why the optimization works but I didn't realize that all worked for lists as inputs which was the use case I cared about (e.g. debugging or lists that already exist). Thanks for the tip though!
|
31

If you want to check if any item in the list violates a condition use all:

if all([x[2] == 0 for x in lista]): # Will run if all elements in the list has x[2] = 0 (use not to invert if necessary)

To remove all elements not matching, use filter

# Will remove all elements where x[2] is 0 listb = filter(lambda x: x[2] != 0, listb)
Improve this answer

2 Comments

You can remove [...] in all(...) since it can then create a generator instead of a list, which not only saves you two characters but also saves memory and time. By using generators, only one item will be calculated at a time (former results will be dropped since no longer used) and if any of them turns out False, the generator will stop calculating the rest.
Note for python3; filter() returns an iterable, not a list. Thus the line would be listb = list(filter(lamba x: x[2] != 0, listb))
8

You could use itertools's takewhile like this, it will stop once a condition is met that fails your statement. The opposite method would be dropwhile

for x in itertools.takewhile(lambda x: x[2] == 0, list) print x
Improve this answer

Comments

1

this way is a bit more flexible than using all():

my_list = [[1, 2, 0], [1, 2, 0], [1, 2, 0]] all_zeros = False if False in [x[2] == 0 for x in my_list] else True any_zeros = True if True in [x[2] == 0 for x in my_list] else False

or more succinctly:

all_zeros = not False in [x[2] == 0 for x in my_list] any_zeros = 0 in [x[2] for x in my_list]
Improve this answer

2 Comments

Couldn't you simply say all_zeros = False in [x[2] == 0 for x in my_list] or even 0 in [x[2] for x in my_list] and correspondingly for any_zeros? I don't really see any remarkable improvement over all().
no, your version - all_zeros = False in [x[2] == 0 for x in my_list] evaluates to False, while mine evaluates to True. If you change it to all_zeros = not (False in [x[2] == 0 for x in my_list]) then it is equivalent to mine. And 0 in [x[2] for x in my_list] is obviously only going to work for any_zeros. But I do like the succinctness of your idea, so I will update my answer
0

Another way to use itertools.ifilter. This checks truthiness and process (using lambda)

Sample-

for x in itertools.ifilter(lambda x: x[2] == 0, my_list): print x
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.