Python: subprocess.popen: read each line of output

Question

I'm having trouble reading my subprocess output by line. The subprocess simply greps the contents of a file against another file. The output, which should be a two column file, prints to stdout just fine. But when I try to read each line, it reads each char followed by \n:

#!/usr/bin/python import sys import getopt import os import subprocess from subprocess import Popen, PIPE, STDOUT inputfile = '' target = '' try: opts, args = getopt.getopt(sys.argv[1:],"s:t:",['servers=', 'target=']) except getopt.GetoptError: print 'getopt failure: exit' sys.exit() for opt, arg in opts: if opt in ("-s", "--servers"): inputfile = arg if opt in ("-t", "--target"): boxwin = arg p1 = subprocess.Popen(["grep -f " + inputfile + " " + target + " | awk '{print $2, $1}'"], stdout=subprocess.PIPE, shell=True) output, error = p1.communicate() print output # this prints normally for line in output: print line # this prints each char of output followed by \n???

Expected output after reading by line:

abc 123 def 456 ghi 789

^^ this will print if I just "print output"

Actual output when using for loop to read each line:

a b c 1 2 3 d e f

...

Any ideas? Thanks.

Community · Accepted Answer · 2017-05-23 11:33:16Z

for c in s: reads one character at a time from a string s (as it should). To get a list of lines from a string instead, you could use .splitlines() method:

lines = output.splitlines()

You don't need to call .communicate() to read the output line by line:

p = subprocess.Popen(cmd, stdout=PIPE) for line in p.stdout: # do something with a line

You could modify the code to handle buffering differently or to enable universal newlines support.

Thanks for the tip. Cut about 5 lines of code out of my script.

jgritty · Accepted Answer · 2022-01-07 21:47:31Z

9

Try the following :

for line in output.splitlines():

instead of:

for line in output:

edited Jan 7, 2022 at 21:47

answered Jan 29, 2014 at 3:47

jgritty

12k3 gold badges41 silver badges62 bronze badges

Collectives™ on Stack Overflow

Python: subprocess.popen: read each line of output

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related