Process stdout or stderr redirected to files during subprocess.check_call

Question

I have this code

fd_log_out = open(path_log_out, 'w') fd_log_err = open(path_log_err, 'w') subprocess.check_call(command, cwd=path, stdout=fd_log_out, stderr=fd_log_err)

I would like to process the stderr during the check_call to trigger an event if something is seen. I have tried to create a subclass of TextIOWrapper and overwrite the write function but it was never called.

What is the called function by subprocess.check_call. The source code is too complex to find it.

Is there any other way to do this? With PIPE maybe ?

Thanks.

The whole point of a redirection is that it's changing where a FD writes to. If you pass an object that has a FD number (that is, where it has a fileno() method) as stdout or stderr, the output from the process goes straight to that FD -- the Python interpreter never sees it at all, and has no opportunity to process it. — Charles Duffy
– Charles Duffy, Commented Jun 20, 2017 at 17:46
The idea was to analyse lines of a wrtie call, check, call super.write. — Benjamin
– Benjamin, Commented Jun 20, 2017 at 17:48
As I said, it's bypassing the interpreter at an OS level; the data isn't exposed to Python at all. (Which is to say, yes, you'd need to use subprocess.PIPE to implement a shim). — Charles Duffy
– Charles Duffy, Commented Jun 20, 2017 at 17:49
Related: How to replicate tee behavior in Python when using subprocess? — wim
– wim, Commented Jun 20, 2017 at 17:59

Charles Duffy · Accepted Answer · 2017-06-20 17:52:43Z

When an object with a fileno() method is passed as an argument to stdout= or stderr= of subprocess.Popen(), then the relevant file descriptor is redirected with fdup2() or a similar syscall.

This means the data is sent directly from the subprocess to the file associated with the FD -- it isn't read by the Python interpreter at all, so the Python interpreter never calls write() on an associated object.

To work around this, you do indeed need to use subprocess.PIPE -- and then have an ongoing process shuffling data on both stdout and stderr (you can't do it one-after-the-other without causing deadlocks) through analysis and to their eventual destinations.

Collectives™ on Stack Overflow

Process stdout or stderr redirected to files during subprocess.check_call

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related