Python Multiprocessing Pickling Confusion in Classes

Question

I have been told that pickling class methods for multiprocessing are not possible with Processes. For some reason though, this code works. I have tried to complete this same task in other applications and my results have been inconsistent. What can I do for this feature to consistently work?

from multiprocessing import Process class test(): def run(self): print("HI") def p(self): p = Process(target=self.run) p.start() a = test() a.p()

I have tried utilizing this functionality with python3 and python and it has worked and not worked for both versions. How do I ensure this works?

What do you mesn with 'worked and not worked'? What did you see? This code would fail on Windows. — azelcer
– azelcer, Commented Jan 31, 2022 at 4:04
I tried both python3 and python and it worked and didn't work for both. Meaning using either version did not make a difference. Why would it not work on windows? Could you elaborate? — Gabe Koleszar
– Gabe Koleszar, Commented Jan 31, 2022 at 13:02
I still don' t get what "it worked and didn't work" mean. It worked once and stopped worked the next time? How did it fail? Did it print anything? See this answer and don't forget to call ` join` — azelcer
– azelcer, Commented Jan 31, 2022 at 21:01

Gabe Koleszar · Accepted Answer · 2022-01-31 18:44:13Z

When starting a new process, Windows and Mac Spawn the process which requires objects to be pickled. On Linux however, the process is Forked which inherits memory from the parent process. If a class contains non-pickle-able objects and you want to run a class method on a separate process, you must use Linux.

Collectives™ on Stack Overflow

Python Multiprocessing Pickling Confusion in Classes

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related