0

Say I have this program printer.py:

#!/usr/bin/env python3 import sys import time sys.stdout.write("STDOUT 1\n") time.sleep(1) sys.stderr.write("STDERR 2\n") time.sleep(1) sys.stdout.write("STDOUT 3\n") time.sleep(1) sys.stderr.write("STDERR 4\n") time.sleep(1) 

It prints to stdout and stderr to produce:

./printer.py STDOUT 1 STDERR 2 STDOUT 3 STDERR 4 

I would like to execute printer.py inside another python script, runner.py, and print in real time both stderr and stdout. The following version of runner.py does not work:

#!/usr/bin/env python3 import sys import subprocess def run_command(command): process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE) while True: output = process.stdout.readline().decode() if output == '' and process.poll() is not None: break if output: print(output.strip()) rc = process.poll() return rc rc = run_command('./printer.py') 

because it prints the stderr lines first in real-time and the stdout lines later all at once:

./runner.py STDERR 2 STDERR 4 STDOUT 1 STDOUT 3 

How can fix it to have the correct order 1, 2, 3, and 4 in real-time? The closer I could get is by using:

rc = run_command('./printer.py 1>&2') 

which is kind of ok, but I wonder whether I could make it do the proper thing and print to stdout and stderr in the same way as printer.py.


sys.stdout.flush() as suggested in comments makes no difference:

#!/usr/bin/env python3 import sys import subprocess def run_command(command): process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE) while True: output = process.stdout.readline().decode() if output == '' and process.poll() is not None: break if output: sys.stdout.write(output.strip() + '\n') sys.stdout.flush() rc = process.poll() return rc rc = run_command('./printer.py') 
./runner.py STDERR 2 STDERR 4 STDOUT 1 STDOUT 3 

The same for print(..., flush=True). Am I doing something wrong?

10
  • Use sys.stdout.flush() to flush the output buffer. Commented Feb 1, 2023 at 19:55
  • @Barmar where do I put sys.stdout.flush()? I tried various places and it makes no difference. Commented Feb 1, 2023 at 20:00
  • After each sys.stdout.write() Commented Feb 1, 2023 at 20:00
  • 1
    Use the unbuffer command to run the child process. Commented Feb 1, 2023 at 20:12
  • 1
    Try run_command('python -u ./printer.py') Commented Feb 1, 2023 at 20:17

1 Answer 1

1

I'm collating comments and add a bit of mine. Credit goes to @Barmar and @MarkSetchell.

In the end, I think I'm going for the following solution:

rc = run_command('PYTHONUNBUFFERED=1 ./printer.py') 

it should do the same as @MarkSetchell's python -u ./printer.py. However, for that I would to explicitly set the path to printer.py and I would rather avoid that. But I'm not sure yet about the pro and cons of each.

unbuffer solution: On my Ubuntu 18 is not installed so I'd rather avoid an additional dependency. As I understand it I would use it as rc = run_command('unbuffer ./printer.py')?

Editing printer.py is not an option for me otherwise adding sys.stdout.flush() after each print or sys.stdout.write should also work.

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.