36

I am working on implementing logging within my Python project and have hit a bit of a snag. I am trying to set up my logging such that the Handlers, and Formatters are all organized into a configuration file. What I am trying to do at the moment is to set up my fileHandler such that it will create a log file that looks something like this: YYYY_MM_DD.log obviously with the Y's representing the year, M's representing the month, and D's representing the day.

This is what I have attempted with my config file:

[loggers] keys=root,MainLogger [handlers] keys=fileHandler, consoleHandler [formatters] keys=logFormatter, consoleFormatter [logger_root] level=DEBUG handlers=fileHandler [logger_MainLogger] level=DEBUG handlers=fileHandler, consoleHandler qualname=MainLogger propagate=0 [handler_consoleHandler] class=StreamHandler level=DEBUG formatter=consoleFormatter args=(sys.stdout,) [handler_fileHandler] class=FileHandler level=DEBUG formatter=logFormatter args=(datetime.now().strftime('%Y_%m_%d.log'), 'a') [formatter_logFormatter] format=%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s [formatter_consoleFormatter] format=%(asctime)s | %(levelname)-8s | %(fillname)s-%(funcName)s-%(lineno)04d | %message)s 

The file I am using to test the configuration is pretty simple:

import logging import logging.config logging.config.fileConfig('logging.conf') logger = logging.getLogger('MainLogger') logger.debug("TEST") 

The specific error I am getting at the moment is:

configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: "%Y_%m_%d.log'), 'a')" 

I've tried changing the %Y, %m, and %d as the error says, but that doesn't fix the problem. How do I go about setting up the config file so that my log files look the way I want them to?

I should note when I change the filename to test.log everything worked fine, so this is the only error I seem to be having with this.

6
  • How are you importing datetime? This could be an error not getting handled correctly due to the common mistake of using datetime.now() instead of datetime.datetime.now(). Commented Aug 1, 2017 at 19:31
  • ......I suppose I am not importing datetime. Though I did just change my config file to datetime.datetime.now() and still got the same error. Would I be importing it in my Python script? Commented Aug 1, 2017 at 19:33
  • Yeah, try putting from datetime import datetime at the top of your script. Commented Aug 1, 2017 at 19:43
  • Hold on. Is that in the logging.conf file? That's not a python script, that just text... Commented Aug 1, 2017 at 19:46
  • Yeah I realize the logging.conf file is just text. That's why I am asking if anyone knows a way to in the configuration file set the Date as the file name. Or is this a case where instead of using a configuration file I should be hard coding it in the script? Commented Aug 1, 2017 at 19:49

13 Answers 13

21

You can't use datetime in a config file, as it doesn't know what it means. You can however add the Filehandler in the python file itself:

import logging.config from datetime import datetime logging.config.fileConfig('aaa.conf') logger = logging.getLogger('MainLogger') fh = logging.FileHandler('{:%Y-%m-%d}.log'.format(datetime.now())) formatter = logging.Formatter('%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s') fh.setFormatter(formatter) logger.addHandler(fh) logger.debug("TEST") 

This way you can set the date as the file name in the handler.

This is the config file, note that you had a typo in the last formatter, you put fillname instead of filename and you forgot ( in message.

[loggers] keys=root,MainLogger [handlers] keys=consoleHandler [formatters] keys=consoleFormatter [logger_root] level=DEBUG handlers=consoleHandler [logger_MainLogger] level=DEBUG handlers=consoleHandler qualname=MainLogger propagate=0 [handler_consoleHandler] class=StreamHandler level=DEBUG formatter=consoleFormatter args=(sys.stdout,) [formatter_consoleFormatter] format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%(lineno)04d | %(message)s 

This Should work just fine.

Sign up to request clarification or add additional context in comments.

Comments

11

Perhaps you can use Python's TimedRotatingFileHandler instead. You can set the interval to create a new log file every day with the date as the suffix.

Documentation--

Note that the current day's log file won't have a date. This file handler only adds the date suffix when a new day starts.

Also, the suffix it uses is "%Y-%m-%d", which is a little different than what you want. But there's a SO question here about how you can alter that.

1 Comment

Not sure if they works in the way I need it to. From my understanding after looking at the links you provided the TimedRotatingFileHandler is for when you are planning on having a program running for multiple days and you want separate log files for each day. In my case my program may be getting used 30 minutes to an hour for each use, but I still need to be able to track the date of when it is used.
6

Using double '%'-characters in the format string in combination with the approach suggested by Abhishek led to a working solution in my case (Python 3.5):

The filehandler in the config file should then look similar to this one:

[handler_fileHandler] class=FileHandler level=DEBUG formatter=defaultFormatter args=(__import__("datetime").datetime.now().strftime('/your_path/your_file_name_%%Y-%%m-%%d_%%H-%%M-%%S.log'), 'a') 

Comments

6

This worked for me.

Update this,

args=(datetime.now().strftime('%Y_%m_%d.log'), 'a') 

with this,

args=(\__import__("datetime").datetime.now().strftime('%Y_%m_%d.log'), 'a') 

Reference (Example no 3): http://python-reference.readthedocs.io/en/latest/docs/functions/eval.html

Comments

4

This post is old, but for anyone still struggling with any such issue of dynamic naming of filenames from logging.conf while calling logging.config.fileConfig() you can pass the variables and use them in the logging configuration file. Eg:

logging.conf

args=('my_log_%(date)s.log','w') 

python.py

import logging.config logging.config.fileConfig('logging.conf', defaults={'date':datetime.now()}) 

You might wanna format the date while using it here.

In the same fashion, any number of variables could be used.

1 Comment

Works for me. Interpolation is better than eval.
3

I wanted to keep all of my logging configurable through the config file and wasn't satisfied with the answers here manually adding the handler through the code. The solution I came up with is to create a handler class inheriting FileHandler which appends the date to the filename parameter before calling the FileHandler constructor:

import os from logging import FileHandler from datetime import datetime class TimestampedFileHandler(FileHandler): def __init__(self, filename, mode='a', encoding=None, delay=False): filename, extension = os.path.splitext(filename) filename = f"{filename}_{datetime.now().strftime('%Y-%m-%d_%H-%M-%S')}{extension}" FileHandler.__init__(self, filename, mode, encoding, delay) 

You can then use the TimestampedFileHandler in your config file and set its level, formatter, and parameters together with the other handlers.

Comments

1

Maybe try changing the name after you've loaded the config file:

from datetime inport datetime logging.config.fileConfig('logging.conf') logging.basicConfig(filename = datetime.now().strftime('%Y_%m_%d.log')) 

3 Comments

Unfortunately this does not work. Once the config file loads, I appear to be unable to make modifications to the file name
Yeah, it was just a guess really, try nofinator's suggestion below: timedrotatingfilehandler
I've been taking a look at it, I'll mark it as an answer if it helps. Thanks for the assistance.
1

My solution is to use f strings. Place f into the 'filename' parameter in your config and add {date.today()} or {datetime.now()}.

import logging from datetime import date, datetime logging.basicConfig(filename=f"C:\\Users\\...\\my_log{date.today()}.log", encoding='utf-8', format='%(asctime)s - %(message)s', level=logging.INFO) 

Comments

0

This uses content from your config file, but does not access the file directly. You create your own filehandler and then add it to the logger.

import logging from datetime import datetime # Create logger. logger = logging.getLogger('MainLogger') logger.setLevel(logging.DEBUG) # Create filehandler with desired filename. fh = logging.FileHandler('{}.log'.format(datetime.now().strftime('%Y_%m_%d'))) fh.setLevel(logging.DEBUG) log_formatter = logging.Formatter('%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s') fh.setFormatter(log_formatter) # Add filehandler to logger. logger.addHandler(fh) 

Note that a (append) is the default mode parameter for a FileHandler.

1 Comment

I assume the import is suppose to be in the Python script? I am still getting the same error :\
0

This also works

from datetime import datetime log_file = str(datetime.utcnow().strftime('%m_%d_%Y_%I_%M_%S')) + '.log' logging.basicConfig(filename=log_file, format='%(levelname)s | %(asctime)s | %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.DEBUG) 

Logfile name:04_30_2018_10_03_01.log

1 Comment

Imagine your program running for two days. Then the file name would remain the same. That would beat the purpose of having a date in the filename.
0

My solution for my python product is to keep the configuration file with a path to a fixed log file such as:

[handler_fileHandler] <br> class=FileHandler <br> level=DEBUG <br> formatter=defaultFormatter <br> args=('../projectnamespace/data/logs/logfile.log',) 

and for each day I will move the log file to older folder logs so by doing that I avoid having a huge log files for later reading....

log_path = str((Path(__file__).parent / 'data' / 'logs' / 'logfile.log').absolute()) log_file_date_created = None if platform.system() == 'Windows': log_file_date_created = datetime.fromtimestamp(os.path.getctime(log_path)) else: stat = os.stat(log_path) try: log_file_date_created = stat.st_birthtime except AttributeError: # running in Linux. No easy way to get creation dates here, # we get when its content was last modified. return stat.st_mtime if log_file_date_created != None and abs(datetime.today().day - log_file_date_created.day) >= 1: file_date = str(log_file_date_created.strftime("%m_%d_%Y")) + '_logfile.log' log_path_new_name = str((Path(__file__).parent / 'data' / 'logs' / 'older' / file_date ).absolute()) os.replace(log_path, log_path_new_name) 

Comments

0

I created class called LoggingService and use it's in other python file

import logging from logging.handlers import RotatingFileHandler from datetime import datetime class LoggingService: def namer(name): now = datetime.now() date_time = now.strftime("%Y-%m-%d") return name[:-1] + date_time +"."+name[-1:] LOG_FORMAT = "%(levelname)s %(asctime)s - %(name)s - %(message)s" log_file = "your inital log file location" rotate_handler = RotatingFileHandler( filename=log_file, mode='a', maxBytes=256, backupCount=8, encoding=None, delay=False ) logging.basicConfig( level=logging.DEBUG, format=LOG_FORMAT, datefmt='%m/%d/%Y %I:%M:%S %p', handlers=[rotate_handler] ) rotate_handler.namer = namer @staticmethod def get_logger(name): return logging.getLogger(name) 

Comments

0

this also worked for me:

import time import logging logging.basicConfig(encoding='utf-8', level=logging.DEBUG) file_handler = logging.FileHandler('mylog-{}.log'.format( time.strftime('%Y%m%d-%H.%M.%S') )) logger.addHandler(file_handler) 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.