Kill children of Python subprocess after subprocess.TimeoutExpired is thrown

Question

I am calling a shell script fom within Python, that spawns multiple child processes. I want to terminate that process and all of its children, if it did not finish after two minutes.

Is there any way I can do that with subprocess.run or do I have to go back to using Popen? Since run is blocking, I am not able to save the pid somewhere to kill the children in an extra command. A short code example:

try: subprocess.run(["my_shell_script"], stderr=subprocess.STDOUT, timeout=120) except subprocess.TimeoutExpired: print("Timeout during execution")

Yes, popen is preferred. Note that killing children of a child is not necessarly possible (depends on my_shell_script). They can be detached. — freakish
– freakish, Commented Jul 4, 2017 at 15:27
In my case I definitely want them killed, since they generate very high load. I will probably revert to use Popen and get a pid for my process in order to kill the process group. — Raptor
– Raptor, Commented Jul 4, 2017 at 15:33

Ben017 · Accepted Answer · 2019-11-07 16:26:09Z

This problem was reported as a bug to the Python developers. It seems to happen specifically when stderr or stdout is redirected. Here is a more correct version of @Tanu's code.

import subprocess as sp try: proc = sp.Popen(['ls', '-l'], stdout=sp.PIPE, stderr=sp.PIPE) outs, errs = proc.communicate(timeout=120) except sp.TimeoutExpired: proc.terminate()

Popen doesn't accept timeout as a parameter. It must be passed to communicate. On Posix OSs, terminate is more gentle than kill, in that it reduces the risk of creating zombie processes.

Tanu · Accepted Answer · 2017-07-04 16:08:27Z

Quoting from the docs:

subprocess.run - This does not capture stdout or stderr by default. To do so, pass PIPE for the stdout and/or stderr arguments.

Don't have to use Popen() if you don't want to. The other functions in the module, such as .call(), .Popen().

There are three 'file' streams: stdin for input, and stdout and stderr for output. The application decides what to write where; usually error and diagnostic information to stderr, the rest to stdout. To capture the output for either of these outputs, specify the subprocess.PIPE argument so that the 'stream' is redirected into your program.

To kill the child process after timeout:

import os import signal import subprocess try: proc = subprocess.Popen(["ls", "-l"], stdout=PIPE, stderr=PIPE, timeout=120) except subprocess.TimeoutExpired: os.kill(proc.pid, signal.SIGTERM)

But how does this help me to kill a child process of my subprocess? As you can see in the example I already capture stderr.
but that doesn't call the process group. in that case proc.terminate() is more portable (windows)

Collectives™ on Stack Overflow

Kill children of Python subprocess after subprocess.TimeoutExpired is thrown

2 Answers 2

Comments

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

4 Comments

Related