1

I tried to make a basic program with multiprocessing in Python but I can't get it to work with the concurrent.futures library. Other posts I have found on the topic did not help me either. The program just gets stuck after starting the futures. I have no idea what the reason for that could be. This is my code:

import concurrent.futures def futureFunction(): print("inside the future") return 0 def myMethod(): print("start pool...") with concurrent.futures.ProcessPoolExecutor() as executor: futs = [] for i in range(4): print("start future", i) futs.append(executor.submit(futureFunction)) for f in concurrent.futures.as_completed(futs): print("done", f.result()) myMethod() 

And this is the output I get:

start pool... start future 0 start future 1 start future 2 start future 3 

Why does this happen? Thanks!

EDIT: I get this Error when running from a .py file (I am using python 3.6)

start pool... start future 0 start future 1 start future 2 start future 3 start pool... start future 0 Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 18, in myMethod start pool... futs.append(executor.submit(futureFunction)) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 466, in submit start pool... self._start_queue_management_thread() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 427, in _start_queue_management_thread self._adjust_process_count() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 446, in _adjust_process_count start future 0 start future 0 p.start() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ prep_data = spawn.get_preparation_data(process_obj._name) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 143, in get_preparation_data Traceback (most recent call last): File "<string>", line 1, in <module> start pool... _check_not_importing_main() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main start pool... start future 0 exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main Traceback (most recent call last): File "<string>", line 1, in <module> start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main Traceback (most recent call last): File "<string>", line 1, in <module> start future 0 start future 0 prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main prepare(preparation_data) exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare start pool... Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main _fixup_main_from_path(data['init_main_from_path']) File "<string>", line 1, in <module> _fixup_main_from_path(data['init_main_from_path']) File "<string>", line 1, in <module> prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path start future 0 File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main Exception in thread Thread-1: Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916, in _bootstrap_inner self.run() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 295, in _queue_management_worker shutdown_worker() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 253, in shutdown_worker call_queue.put_nowait(None) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\queues.py", line 129, in put_nowait return self.put(obj, False) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\queues.py", line 83, in put raise Full queue.Full Traceback (most recent call last): File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 22, in myMethod print("done", f.result()) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\_base.py", line 425, in result return self.__get_result() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\_base.py", line 384, in __get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending. C:\Users\pcuser\Desktop>test.py start pool... start future 0 start future 1 start future 2 start future 3 start pool... start future 0 Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="__mp_main__") File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 18, in myMethod futs.append(executor.submit(futureFunction)) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 466, in submit self._start_queue_management_thread() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 427, in _start_queue_management_thread self._adjust_process_count() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 446, in _adjust_process_count p.start() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ start pool... prep_data = spawn.get_preparation_data(process_obj._name) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main start pool... start future 0 is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. Traceback (most recent call last): File "<string>", line 1, in <module> start future 0 File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main Traceback (most recent call last): File "<string>", line 1, in <module> prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare exitcode = _main(fd) start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main _fixup_main_from_path(data['init_main_from_path']) prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path start pool... run_name="__mp_main__") File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare start future 0 pkg_name=pkg_name, script_name=fname) _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code start future 0 Traceback (most recent call last): run_name="__mp_main__") mod_name, mod_spec, pkg_name, script_name) File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code pkg_name=pkg_name, script_name=fname) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main Traceback (most recent call last): File "<string>", line 1, in <module> exec(code, run_globals) File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) Exception in thread Thread-1: Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916, in _bootstrap_inner self.run() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 295, in _queue_management_worker shutdown_worker() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 253, in shutdown_worker call_queue.put_nowait(None) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\queues.py", line 129, in put_nowait return self.put(obj, False) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\queues.py", line 83, in put raise Full queue.Full Traceback (most recent call last): File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 22, in myMethod print("done", f.result()) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\_base.py", line 425, in result return self.__get_result() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\_base.py", line 384, in __get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending. C:\Users\pcuser\Desktop>test.py start pool... start future 0 start future 1 start future 2 start future 3 start pool... start future 0 Traceback (most recent call last): File "<string>", line 1, in <module> start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare start future 0 _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path Traceback (most recent call last): start pool... File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main run_name="__mp_main__") File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare pkg_name=pkg_name, script_name=fname) start pool... start future 0 File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 18, in myMethod Traceback (most recent call last): futs.append(executor.submit(futureFunction)) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 466, in submit _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path start future 0 run_name="__mp_main__") File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path self._start_queue_management_thread() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 427, in _start_queue_management_thread self._adjust_process_count() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 446, in _adjust_process_count File "<string>", line 1, in <module> Traceback (most recent call last): pkg_name=pkg_name, script_name=fname) p.start() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main self._popen = self._Popen(self) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main mod_name, mod_spec, pkg_name, script_name) return _default_context.get_context().Process._Popen(process_obj) exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 322, in _Popen start pool... prepare(preparation_data) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare exec(code, run_globals) File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 18, in myMethod return Popen(process_obj) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\popen_spawn_win32.py", line 33, in __init__ _fixup_main_from_path(data['init_main_from_path']) start future 0 futs.append(executor.submit(futureFunction)) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare prep_data = spawn.get_preparation_data(process_obj._name) start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 466, in submit Traceback (most recent call last): _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 143, in get_preparation_data start pool... run_name="__mp_main__") self._start_queue_management_thread() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 427, in _start_queue_management_thread start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path start future 0 _check_not_importing_main() start pool... start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path File "<string>", line 1, in <module> self._adjust_process_count() start future 0 run_name="__mp_main__") File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main Traceback (most recent call last): start future 0 pkg_name=pkg_name, script_name=fname) start future 0 start future 0 start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 446, in _adjust_process_count start pool... File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 263, in run_path Traceback (most recent call last): is not going to be frozen to produce an executable.''') File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code Traceback (most recent call last): Traceback (most recent call last): Traceback (most recent call last): exitcode = _main(fd) p.start() start future 0 pkg_name=pkg_name, script_name=fname) File "<string>", line 1, in <module> RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable.start future 0 File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main mod_name, mod_spec, pkg_name, script_name) File "<string>", line 1, in <module> File "<string>", line 1, in <module> File "<string>", line 1, in <module> File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 96, in _run_module_code Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 105, in spawn_main prepare(preparation_data) self._popen = self._Popen(self) mod_name, mod_spec, pkg_name, script_name) File "<string>", line 1, in <module> exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 114, in _main exec(code, run_globals) File "<string>", line 1, in <module> exitcode = _main(fd) exitcode = _main(fd) exitcode = _main(fd) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 225, in prepare File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code Exception in thread Thread-1: Traceback (most recent call last): File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\threading.py", line 916, in _bootstrap_inner self.run() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 295, in _queue_management_worker shutdown_worker() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\process.py", line 253, in shutdown_worker call_queue.put_nowait(None) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\queues.py", line 129, in put_nowait return self.put(obj, False) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\multiprocessing\queues.py", line 83, in put raise Full queue.Full Traceback (most recent call last): File "C:\Users\pcuser\Desktop\test.py", line 24, in <module> myMethod() File "C:\Users\pcuser\Desktop\test.py", line 22, in myMethod print("done", f.result()) File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\_base.py", line 425, in result return self.__get_result() File "C:\Users\pcuser\AppData\Local\Programs\Python\Python36\lib\concurrent\futures\_base.py", line 384, in __get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending. 

EDIT 2: When I change the method call to:

if __name__ == '__main__': myMethod() 

the problem is fixed for when I run the program from a .py file. But the problem still exists when I try to run the code inside visual studio.

2
  • Is the error still consistent with EDIT2? Commented Apr 21, 2021 at 18:35
  • Yes, I still haven't figured out how to solve it, the program just hangs up and nothing happens. Commented Apr 21, 2021 at 21:45

1 Answer 1

8

If you just want to solve this problem. You can try to use concurrent.futures.ThreadPoolExecutor(max_workers) in place of concurrent.futures.ProcessPoolExecutor().

The default setting of max_workers is based on the number of CPUs. You can check the documentation of the ThreadPoolExecutor().

If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.