Using "+=" in array of Counters resulted in an unexpected behavior, Python

Question

I am using Python 3.4.1 and I was wondering about the following situation:

Given an array of counters

cnt = [Counter()] * n

I want to add some items in a specific position, so I do

cnt[i] += Counter(x)

For the construction "+=", I was trying to do

cnt[i] = cnt[i] + Counter(x)

But, instead of what I expected, I received something equivalent to

for i in range(0, n): cnt[i] = cnt[i] + Counter(x)

In other words, it added all my counters in the array.

Is this behavior (add every item of the array) common in Python? Am I interpreting anything wrong?
There is a correct/easy/safe way to write what I desired?
Is this a bug of the version?

An short example:

from collections import Counter text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit." cnt = [Counter()] * 2 i = 0 for c in text: cnt[i] += Counter(c) # cnt[i] = cnt[i] + Counter(c) i = (i+1) % 2 for i in range(0, 2): print(cnt[i], i)

Output:

Counter({' ': 7, 'i': 6, 'e': 5, 't': 5, 'o': 4, 's': 4, 'm': 3, 'r': 3, 'c': 3, 'u': 2, 'p': 2, 'a': 2, 'l': 2, 'd': 2, 'n': 2, '.': 1, 'g': 1, 'L': 1, ',': 1}) 0 Counter({' ': 7, 'i': 6, 'e': 5, 't': 5, 'o': 4, 's': 4, 'm': 3, 'r': 3, 'c': 3, 'u': 2, 'p': 2, 'a': 2, 'l': 2, 'd': 2, 'n': 2, '.': 1, 'g': 1, 'L': 1, ',': 1}) 1

Expected output:

Counter({'t': 4, 'i': 3, 'r': 3, 's': 2, 'e': 2, 'm': 2, 'c': 2, 'n': 2, 'a': 2, 'l': 2, ',': 1, 'd': 1, ' ': 1, 'L': 1}) 0 Counter({' ': 6, 'o': 4, 'i': 3, 'e': 3, 's': 2, 'u': 2, 'p': 2, '.': 1, 't': 1, 'g': 1, 'd': 1, 'm': 1, 'c': 1}) 1

All the elements of the cnt array refer to the same counter. You didn't make a copy of it when you used the * operator. — Barmar
– Barmar, Commented Sep 14, 2014 at 4:09
Are you sure? I already used * other times and it works well, ie, make different copies. — MFS
– MFS, Commented Sep 14, 2014 at 4:13
@MFS: No it doesn't. You might have thought it did if you were using immutable objects, but even then, you had multiple references to one object. Immutables just handle things like += differently. — user2357112
– user2357112, Commented Sep 14, 2014 at 4:15
Specifically, if you have an array of numbers, it works because they're immutable. So it updates the array reference rather than the shared object. — Barmar
– Barmar, Commented Sep 14, 2014 at 4:17
@user3885927: That's because support for __iadd__ was added to Counter in a later Python version. I don't remember which, but 3.4 should have it. — user2357112
– user2357112, Commented Sep 14, 2014 at 4:24

Michael0x2a · Accepted Answer · 2014-09-14 04:12:11Z

When you do cnt = [Counter()] * n, what you're doing is creating a single counter, then making every element in your list point to that counter. You're not creating n Counters, you're creating a single one.

This is because in Python, everything is stored by reference (sort of. More info here). You've essentially duplicated the reference to the counter object n times, not the counter itself.

That means that doing cnt[i] += Counter(x) will modify the underlying counter, making it appear like the entire list changed.

To fix this, try doing something like the following:

cnt = [Counter() for i in range(n)]

Now, you're genuinely creating n different counters (because you call the constructor n times) and will get the expected behavior.

Ok, I understand it. But, do you know why when I change the line with the comment (in the example I gave) for the comment, I reach the expected output?
@MFS: a += b isn't equivalent to a = a + b. += requests that the object perform the operation by modifying its state, rather than by returning a new object, if it supports modification.

Collectives™ on Stack Overflow

Using "+=" in array of Counters resulted in an unexpected behavior, Python

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related