I'm trying to find out a way in python to redirect the script execution log to a file as well as stdout in a pythonic way. Is there any easy way of achieving this?
8 Answers
Use logging module (http://docs.python.org/library/logging.html):
import logging logger = logging.getLogger('scope.name') file_log_handler = logging.FileHandler('logfile.log') logger.addHandler(file_log_handler) stderr_log_handler = logging.StreamHandler() logger.addHandler(stderr_log_handler) # nice output format formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') file_log_handler.setFormatter(formatter) stderr_log_handler.setFormatter(formatter) logger.info('Info message') logger.error('Error message') 
Comments
I came up with this [untested]
import sys class Tee(object): def __init__(self, *files): self.files = files def write(self, obj): for f in self.files: f.write(obj) f.flush() # If you want the output to be visible immediately def flush(self) : for f in self.files: f.flush() f = open('out.txt', 'w') original = sys.stdout sys.stdout = Tee(sys.stdout, f) print "test" # This will go to stdout and the file out.txt #use the original sys.stdout = original print "This won't appear on file" # Only on stdout f.close() print>>xyz in python will expect a write() function in xyz. You could use your own custom object which has this. Or else, you could also have sys.stdout refer to your object, in which case it will be tee-ed even without >>xyz.
11 Comments
Vijay
Awesome. This works the way I expected.. since i'm new to programming, trying to understand a bit.. how is the class Tee is called when using the command Tee(sys.stdout,f) as the class def involves only an object (could be a file object) ? please advice
eacousineau
I find this especially helpful for writing quick LaTeX generators. @Thrustmaster I made another small modification, posted a Gist. Thank you for posting this!
Eliezer Miron
This works great, except when using subprocess. In my main function I do this, then call a function outside main, which calls subprocess, and the output from the subprocess is not written to the file (though it is written to stdout, as normal)
Erik Anderson
Couldn't you restore
sys.stdout with sys.__stdout__ instead of saving it out to a variable?
Alfredo Capobianchi
At the
f.close() instruction, in the end of my script, I get the following error: Exception ignored in: <__main__.Tee object at 0x7f715e77e760> Traceback (most recent call last): File "constellation_test.py", line 40, in flush f.flush() ValueError: I/O operation on closed file. Why? |
I just want to build upon Serpens answer and add the line:
logger.setLevel('DEBUG') This will allow you to chose what level of message gets logged.
For example in Serpens example,
logger.info('Info message') Will not get recorded as it defaults to only recording Warnings and above.
More about levels used can be read about here
Comments
Probably the shortest solution:
def printLog(*args, **kwargs): print(*args, **kwargs) with open('output.out','a') as file: print(*args, **kwargs, file=file) printLog('hello world') Writes 'hello world' to sys.stdout and to output.out and works exactly the same way as print().
Note: Please do not specify the file argument for the printLog function. Calls like printLog('test',file='output2.out') are not supported.
3 Comments
Lei Yang
is there any performance drawbacks? i mean open/close file repeatedly?
Markus Dutschke
depends on your use case. if you put
printLog into an often repeated loop this will have strong impact on performance. But putting print there is not a good idea, either! What might be an idea is to add some buffer an just write 100 lines at once in the log file. This somehow might be against the idea of your logger
Alaa M.
To specify the filename, just wrap
printLog with another function that takes a filename: def print_log(msg, filename): and inside it define your printLog (can name it _print_log to indicate privacy), and inside print_log, call _print_log(msg). This way your inner function also have access to the filename parameter from the outer print_log function. So it can use it instead of output.out.Here's a small improvement that to @UltraInstinct's Tee class, modified to be a context manager and also captures any exceptions.
import traceback import sys # Context manager that copies stdout and any exceptions to a log file class Tee(object): def __init__(self, filename): self.file = open(filename, 'w') self.stdout = sys.stdout def __enter__(self): sys.stdout = self def __exit__(self, exc_type, exc_value, tb): sys.stdout = self.stdout if exc_type is not None: self.file.write(traceback.format_exc()) self.file.close() def write(self, data): self.file.write(data) self.stdout.write(data) def flush(self): self.file.flush() self.stdout.flush() To use the context manager:
print("Print") with Tee('test.txt'): print("Print+Write") raise Exception("Test") print("Print") 
Comments
You should use the logging library, which has this capability built in. You simply add handlers to a logger to determine where to send the output.
Comments
The easiest solution is to redirect the standard output. In your python program file use the following:
if __name__ == "__main__": sys.stdout = open('file.log', 'w') #sys.stdout = open('/dev/null', 'w') main() Any std output (e.g. the output of print 'hi there') will be redirected to file.log or if you uncomment the second line, any output will just be suppressed.
1 Comment
Vijay
This directs the output only to a file.. I want the logs to be displayed both to the file and to the console...
Create an output file and custom function:
outputFile = open('outputfile.log', 'w') def printing(text): print(text) if outputFile: outputFile.write(str(text)) Then instead of print(text) in your code, call printing function.
printing("START") printing(datetime.datetime.now()) printing("COMPLETE") printing(datetime.datetime.now()) 
script | tee logfilename