2

I am using subprocess in Python to call an external program on WINDOWS. I control the process with ThreadPool so that I can limit it to max 6 processes at the same time, and new process continuously began when one was done.

Codes as below:

### some codes above ### Code of Subprocess Part from multiprocessing.pool import ThreadPool as Pool def FAST_worker(file): p = subprocess.Popen([r'E:/pyworkspace/FAST/FAST_RV_W64.exe', file], cwd = r'E:/pyworkspace/FAST/', shell = True) p.wait() # List of *.in filenames FAST_in_pathname_li = [ '334.in', '893.in', '9527.in', ... '114514.in', '1919810.in', ] # Limit max 6 processes at same time with Pool(processes = 6) as pool: for result in pool.imap_unordered(FAST_worker, FAST_in_pathname_li): pass ### some codes below

I got problem when the external program unexpectedly terminated and showed error message pop-up. Though the other 5 processes still kept going, the whole progress will finally get stuck at the "subprocess part" and couldn't go forward anymore. (unless I came to my desk and manually clicked "Shut down the program")

What I want to know is how can I avoid the pop-up and make the whole script process keep going, like bypass the error message or something, rather than manual click, in order to avoid wasting time.

Improve this question

1 Answer 1

Reset to default
2

Since we don't know enough about the program FAST_worker is calling, I'll assume you already checked there isn't any "kill on error" or "quiet" mode that would be more convenient to use in a script.

My two cents: maybe you can setup a timeout on the worker execution, so that a stuck process is killed automatically after a certain delay.

Building on the snippet provided here, here is a draft:

from threading import Timer def FAST_worker(file, timeout_sec): def kill_proc(): """called by the Timer thread upon expiration""" p.kill() # maybe add task to list of failed task, for tracability p = subprocess.Popen([r'E:/pyworkspace/FAST/FAST_RV_W64.exe', file], cwd = r'E:/pyworkspace/FAST/', shell = True) # setup timer to kill the process after a timeout timer = Timer(timeout_sec, kill_proc) try: timer.start() stdout, stderr = p.wait() finally: timer.cancel()

Note that there are also GUI automation libraries in python that can do the clicking for you, but that is likely to be more tedious to program:

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I've tried applying code above into my script, but the behavior of the process became weird. (sorry for not know how to explain) Maybe I'll contact with the writer of the FAST_RV_W64.exe program. Still thanks for your help.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.