1

I have an executable function called BOB.exe which prints an insane amount of text to stdout with only short pauses. BOB also has a habit of freezing so I wrote a python monitor function which, using the subprocess module, calls the BOB executable, directs it to a temporary file and watches the temp file's size to see if it has crashed. This is my current solution:

#!/usr/bin/python from subprocess import Popen import tempfile, time def runBOB(argsList): # Create a temporary file where BOB stdout will be piped BOBout = tempfile.NamedTemporaryFile() BOBoutSize = 0 # Start the subprocess of BOB BOBsp = Popen(argsList, stdout=BOBout) while True: # See if subprocess has finished if BOBsp.poll() is not None: BOBout.close() # Destroy the temp file return 0 # if the size of the stdout file has increased, BOB.exe is still running BOBoutSizeNew = os.path.getsize(BOBout.name) if BOBoutSizeNew > BOBoutSize: BOBoutSize = BOBoutSizeNew else: # if not, kill it BOBsp.kill() BOBout.close() # Destroy the temp file return 1 # Check every 10 seconds time.sleep(10)

However, this is incredibly slow and I think the writing to file is the cause. Is there a more efficient way to do this such as watching the stdout stream and then immediately sending it to Null? Anything that cuts down on the megabytes of printing being done would probably help. Is there another way of seeing if the exe has crashed? I should probably note that I don't care about the stdout, it is all going to be ignored anyway

Thank you for your help!

Improve this question
0

1 Answer 1

Reset to default
2

You can use stdout=subprocess.PIPE to tell subprocess to give you ability to read the output of the subprocess without storing it to a file. The tricky part is to do so asynchronously, so as not to deadlock when BOB.exe freezes. An easy way to do so is with a helper thread; although Python has a bad reputation with threads, this is actually a good use case for threading where GIL doesn't get in the way.

Simply create a helper thread that does nothing except reading the output from the file handle corresponding to Bob's output. The helper thread immediately discards the output and increments a byte counter. The main thread implements exactly the same logic you had previously, but using the in-memory counter instead of re-examining the file size. When Bob finishes or is killed by the main thread, the helper thread will receive EOF and exit.

Here is an untested implementation of the above:

#!/usr/bin/python import subprocess import threading import time import os bytes_read = 0 def readBOB(pipe): global bytes_read bytes_read = 0 while True: # Wait for some data to arrive. This must use os.read rather # than pipe.read(1024) because file.read would block us if less # than 1024 bytes of data arrives. (Reading one byte at a time # with pipe.read(1) would work, but would be too slow at # consuming large amounts of data.) s = os.read(pipe.fileno(), 1024) if not s: return # EOF # we are the only writer, so GIL serves as the lock bytes_read += len(s) def runBOB(argsList): # Start the subprocess of BOB BOBsp = subprocess.Popen(argsList, stdout=subprocess.PIPE) thr = threading.Thread(target=readBOB, args=(BOBsp.stdout,)) thr.start() old_bytes_read = -1 while True: # See if subprocess has finished if BOBsp.poll() is not None: return 0 # if the size of the stdout has increased, BOB.exe is still running new_bytes_read = bytes_read if new_bytes_read > old_bytes_read: old_bytes_read = new_bytes_read else: # if not, kill it (readBOB will exit automatically) BOBsp.kill() return 1 # Check every 10 seconds time.sleep(10)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thank you very much for that! Very understandable too. A single question since I am not too familiar with pythons threading: Would readBOB be constantly processing in the background or is there some limit to how often the loop runs? Would it be wise to for efficiency reasons?
@Goose readBOB is not busy-looping, if that's what's worrying you. os.read will block when there is no data to read, so readBOB will only spend as much CPU as needed to read (and discard) 1k chunks of data produced by Bob, which should be negligible.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.