Why does python's SharedMemory seem to initialize arrays to zeros

Question

I'm initializing a SharedMemory in python to be shared between multiple processes and I've noticed that it always seems to be filled with zeros (which is fine), but I don't understand why this is occurring as the documentation doesn't state there is a default value to fill the memory with.

This is my test code, opened in two seperate power shells, shell 1:

import numpy as np from multiprocessing.shared_memory import SharedMemory def get_array_nbytes(rows, cols, dtype): array = np.zeros((rows, cols), dtype=dtype) nbytes = array.nbytes del array return nbytes rows = 10000000 depths_columns = 18 array_sm = SharedMemory(create=True, size=get_array_nbytes(rows, depths_columns, np.float32), name='array_sm')

shell 2:

from multiprocessing.shared_memory import SharedMemory import numpy as np array_sm = SharedMemory("depths_array") array = np.ndarray((rows, 18), dtype=np.float32, buffer=array_sm.buf)

now in the second shell you can follow this up with:

array[0] array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)

or

np.where(array != 0) (array([], dtype=int64), array([], dtype=int64))

Is this behavior always going to be the case or is this a fluke? Is there some sort of undocumented initialization to zero happening in the background?

tdelaney · Accepted Answer · 2021-11-17 17:35:54Z

This is operating system dependent. Python doesn't initialize the memory - it just takes the virtual memory address offered by the operating system. On posix systems it uses shm_open, while on Windows its CreateFileMapping. On linux and windows, these calls guarantee that memory is initialized to zero.

It would be a security leak to let the application see whatever left over data happens to be in the RAM from the previous user, so it needs to be filled with something. But this isn't a guarantee from python and its possible that some operating systems (embedded OS perhaps) don't do things that way.

Collectives™ on Stack Overflow

Why does python's SharedMemory seem to initialize arrays to zeros

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related