Python - a subprocess writing to stdin so that the main program can read it from the stdin

Question

I have a python program which takes input from stdin. Now, I've to write another program and call it as a sub process so that every time I start this subprocess it should read data from another text file and write it to stdin and then my main program reads it from stdin and uses it.

out.py:

from subprocess import Popen, PIPE, STDOUT import os import sys def read(): p = Popen(['python', 'in.py'], stdout=PIPE, stdin=PIPE, stderr=STDOUT) out = p.communicate() print out[0] def main(): read() if __name__ == "__main__": main()

in.py:

import sys,os import subprocess def main(): f = open("ad.txt","r") for line in f: print line if __name__ == "__main__": main()

Basically my problem here is the in.py reads the whole file and prints into STDOUT which I don't want and rather it should communicate one character once. If the ad.txt is like :

asdfgh

I should get "a" then "s" then "d" then "f" .. so on.. one by one character everytime I call the function read the next character from the file should be read. That's the thing! Phew!! Please help me and I got a heck of a work to do! Thanks in advance :D

Why not just have one program do all the work? You can still split that across several files. You know how modules work in Python, yes? — Karl Knechtel
– Karl Knechtel, Commented Dec 30, 2011 at 7:38
@KarlKnechtel : I'm writing a VM so I shouldn't do it.. I mean the VM specification is like it should use another program to do it as I've told above! — VoodooChild92
– VoodooChild92, Commented Dec 30, 2011 at 7:47

Eser Aygün · Accepted Answer · 2011-12-30 08:35:54Z

The first program should read from a file (or stdin) and write to stdout (not stdin). Similarly, the second program should read from stdin and write to stdout. Now you can glue them together in the command line using the pipe symbol:

python first.py | python second.py

That's it! None of the programs has to be aware of the other; that's the beauty of using "pipelining".

Addendum: Shell pipelining works over a buffer. The shell listens to the first program and fills the buffer with its output. Simultaneously, the second program reads from the buffer as long as there is something to read, otherwise it waits idly. The communication is simultaneous and requires only a fixed size of memory.

Example code:

# first.py import sys for line in open("input.txt"): processed_line = process_line(line) sys.stdout.write(processed_line) # second.py import sys for line in sys.stdin: processed_line = process_line(line) sys.stdout.write(processed_line)

These examples work over lines (which is better if you are working with text files). You could easily do the same byte by byte.

That's useful! But do check out my edit .. i added the code.. Let me know if you got any changes in your suggestion!! Please !! Thanks !
No problem! I updated the answer. As I explained in the answer, you don't need to create subprocesses inside the Python code. Let them read from stdin and write to stdout as usual. The shell will do the rest.
Simple as a gunshot, mate! Kinda expected to go a bit technical but yes it's great enough for me!! Thanks again!!

Collectives™ on Stack Overflow

Python - a subprocess writing to stdin so that the main program can read it from the stdin

out.py:

in.py:

1 Answer 1

3 Comments

Hot Network Questions

Collectives™ on Stack Overflow

out.py:

in.py:

1 Answer 1

3 Comments

Related