Reading stdin line at a time in Python

Question

I am trying to test a particular chat program by sending a series of commands and capturing the output.

How can I pass:

sleep 2 echo test sleep 2 echo test1

I have tried this:

(sleep 2; echo test; sleep 2; echo test1) | python3 test.py

but it just prints the first part while for the second I get nothing. It enters into an endless loop instead.

the python program's code is:

import sys, select while True: socket_list = [sys.stdin] read_sockets, write_sockets, error_sockets = select.select(socket_list, [], []) for sock in read_sockets: message = sys.stdin.readline() sys.stdout.write("> %s: ") sys.stdout.flush()

I should mentioned that this is not the complete program but it is the part where it helps recreate the exact same effect.

Shot in the dark, but what about { sleep 2; echo test; sleep 2; echo test1 } and maybe redirect it to python instead of pipe? — jesse_b
– jesse_b, Commented Aug 30, 2017 at 23:36
date +%s; (sleep 2; echo test; sleep 2; echo test1) | perl -pe '$_=time()." ".$_' shows both test strings and the two-second delays for me. — thrig
– thrig, Commented Aug 30, 2017 at 23:40
You need to show us how you are sending and receiving. It sounds like a problem for SO, but it's hard to tell. — Jeff Schaller
– Jeff Schaller, Commented Aug 31, 2017 at 1:06
Assuming, this is a reduced example for something that will need multiple sockets, use Twisted (or a similar library). Do not implement the whole event dispatch system yourself. — o11c
– o11c, Commented Aug 31, 2017 at 15:12

sebasth · Accepted Answer · 2017-08-31 18:20:14Z

Your example goes to loop because after echo test1 stdin is closed and read() always returns empty. Also you shouldn't use another blocking call with select. If you would use it on more than one object, you could still be blocked by readline from the previous event.

For reading a line at time from stdin you do not need to use select.select(). Possibly the simplest way to achieve that:

for line in sys.stdin: print(line)

If you want to use either stdin or a list of files as arguments to your program, Python's fileinput module is out of the box solution.

In case you want/need to use select.select() anyways:

import os, sys, select buffer = "" while True: select.select([sys.stdin.fileno()], [], []) read = os.read(sys.stdin.fileno(), 512) # empty read: EOF if len(read) == 0: # buffer might not be empty if len(buffer) > 0: sys.stdout.write(buffer + "\n") break # find newlines parts = read.split("\n") buffer += parts.pop(0) while len(parts) > 0: sys.stdout.write(buffer + "\n") buffer = parts.pop(0)

When an object becomes ready, select.select unblocks. The only object in this case is stdin. With more than one object you need to check the return value of select.select to find out which one is ready. os.read() is used to do non-blocking read from stdin (alternatively: it's possible to use stdin.read(1) reading a character at a time). read.split("\n") is used for finding newlines, note that it is possible a single read yields more than one line.

Excellent response and thank you for sending the example using select.select. Indeed after the last echo there is multiple b'' sent to stdin so checking for an empty read and exiting fixed the issue.

score 1 · Accepted Answer · 2017-08-31 17:10:21Z

After running strace on cat, I created this:

import sys while True: data = sys.stdin.read() if not len(data): break sys.stdout.write(data)

Command

(sleep 2; echo test; sleep 2; echo test1) | python3 test.py

Output

test test1

This solution relies on this fact:

If the end of the file has been reached, f.read() will return an empty string ('').

See Methods of File Objects.

If you need a version that uses select:

import os import select import sys while True: rlist = [sys.stdin.fileno()] wlist = [] xlist = [] rlist, wlist, xlist = select.select(rlist, wlist, xlist) if sys.stdin.fileno() in rlist: data = os.read(sys.stdin.fileno(), 4096) if not len(data): break sys.stdout.write(data)

Collectives™ on Stack Overflow

Reading stdin line at a time in Python

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related