2

Why is img_list smaller than compressed even though is smaller?

Input

print(sys.getsizeof(img_list[0:1])) print(img_list[0:1]) print(sys.getsizeof(compressed[0:2])) print(compressed[0:2]) print(sys.getsizeof(img_list)) print(sys.getsizeof(compressed)) img_arr = np.asanyarray(img) print(img_arr.shape) comp_arr = np.asarray(compressed) print(comp_arr.shape)

Output

72 [[[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [19, 19, 19], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]] 80 [[12, [0, 0, 0]], [1, [19, 19, 19]]] 256 2536 (24, 24, 3) (306, 2)
Improve this question
9
  • 2
    you haven't show then length of your lists here. Can you provide a minimal reproducible example? Commented Dec 11, 2019 at 18:03
  • @Error-SyntacticalRemorse I don't think that's what is going on here. Again, the OP hasn't actually provided the len of either list. I suspect that img_list is actually about 1/10 the length of compressed. Commented Dec 11, 2019 at 18:08
  • 1
    @Error-SyntacticalRemorse but the examples with slices are consistent, the smaller one has the smaller memory footprint. The slices have length 1 and 2 respectively... Commented Dec 11, 2019 at 18:10
  • I will provide the full lists instead Commented Dec 11, 2019 at 18:10
  • I converted them to numpy arrays and printed the shape of them for clarification Commented Dec 11, 2019 at 18:15

1 Answer 1

Reset to default
1

sys.getsize() is deceiving. It returns the size of an object; however, it does not consider the size of that objects attributes. In other words, it does not recursively go through the object you are getting the size of. Ultimately you need to do that on your own:

import sys l1 = [[[0, 0, 0], [0, 19, 19], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 13, 0], [0, 0, 0], [19, 19, 19], [110, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 12, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]]] l2 = [[12, [0, 0, 0]], [1, [19, 19, 19]]] def get_size(obj, seen=None): """Recursively finds size of objects""" size = sys.getsizeof(obj) if seen is None: seen = set() obj_id = id(obj) if obj_id in seen: return 0 # Important mark as seen *before* entering recursion to gracefully handle # self-referential objects seen.add(obj_id) if isinstance(obj, dict): size += sum([get_size(v, seen) for v in obj.values()]) size += sum([get_size(k, seen) for k in obj.keys()]) elif hasattr(obj, '__dict__'): size += get_size(obj.__dict__, seen) elif hasattr(obj, '__iter__') and not isinstance(obj, (str, bytes, bytearray)): size += sum([get_size(i, seen) for i in obj]) return size print(get_size(l1)) print(get_size(l2))

Output:

5280 564

Reference: Measure the Real Size of Any Python Object

What you were doing was basically:

sys.getsizeof([[]])

and

sys.getsizeof([[], []]) # This is bigger
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks a lot, really helpful
I still have some issues though. When I run them through the function. When I run the img_arr and the comp_arr, I get the results 582 for the img_arr and 5484 for the comp_arr. When I run the list they are almost the same size. I suppose that is the corrects values, but why am I getting them?
@DenFulaAnkungen is it possible several elements of your array are pointing to the same object? like a = [[0,0,0],[0,0,0],[0,0,0]] will have 3 distinct lists which happen to have the same elements where as sub = [0,0,0]; b = [sub,sub,sub] has 3 references to the same list and therefore less space.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.