How read in realtime an output process with python?

Question

I need to realise a python script who read the ouput of an other process in real time line by line. Obviously I've ever try to use "subprocess" to get the output process with stdout. But i can't get that output in real time, indeed every times python script return the output in several chunk few time after the process launch. For exemple i use this python script :

import subprocess as sub import sys proc = sub.Popen(["python", "bin/read_test.py"], stdout = sub.PIPE, bufsize = 1) while True: if proc.poll() is None: line = proc.stdout.read() line = line.decode("utf-8") print(line, flush = True) sys.stdout.flush() else: proc.stdout.close() break

read_test.py script to be read :

from time import sleep for i in range(5): print(i) sleep(1)

I've try a lot of methods with "readline()" with for loops but the issue still the same.

Moreover I don't want to use "communicate" because it's a blocking method.

Thanks for your help,

Possible duplicate of read subprocess stdout line by line

pltrdy
– pltrdy

2017-02-20 14:15:19 +00:00
Commented Feb 20, 2017 at 14:15 — pltrdy
– pltrdy, Commented Feb 20, 2017 at 14:15

Jean-François Fabre · Accepted Answer · 2017-02-20 14:17:04Z

Problem is that you're trying to read stdout fully.

Since python sees that process is still running, it waits until process ends so process output is complete.

To do what you want you probably want to read the output line by line using line = proc.stdout.readline()

You probably have to change your loop to read line, and stop when process ends

proc = sub.Popen(["python", "bin/read_test.py"], stdout = sub.PIPE, bufsize = 1) while True: line = proc.stdout.readline() if line: line = line.decode("utf-8") print(line) sys.stdout.flush() if proc.poll() is not None: # process ends break proc.wait()

Also that's not enough: you have to flush the output on the read_test.py side to make sure that the emitter actually sends the lines (when output is redirected, it's not sure). Example program:

import time,sys for i in range(5): print(i) sys.stdout.flush() # not flushing: you get all the lines at the end of the process time.sleep(1)

I've connected both programs, and got 1 sequenced output (1,2,3,4,5) each second

Ok, add the flush in the read_test.py have solved my issue, thank you.

Collectives™ on Stack Overflow

How read in realtime an output process with python?

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related