Logging between modules in Python

Question

I am learning logging with the standard Python library documentation and it seems I am missing something trivial. I want to have logging in multiple modules but configure each instance only in the main file. I think that what I am doing wrong is that basicConfig only works for the functions that control the root logger, but I am confused.

Could you point me in the right direction? Thanks in advance.

# main.py import logging import module_log logging.basicConfig(level=logging.WARNING) logger = logging.getLogger(__name__) def test1(): logger.warning("This one is not showing in log but that is what I expect") module_log.test2() if __name__ == "__main__": test1() # module.py import logging logger = logging.getLogger(__name__) def test2(): logger.warning("This one shouldn't be showing in log but it is")

EDIT 1: Updated the code to better illustrate the problem.

EDIT 2: I think I found the source of the problem, even thought I don't fully understand why. I was not executing the main.py directly, I was calling it from within the Maya(3d software) interpreter (but the same happens with the Python interpreter) If you run:

>>> import main_log >>> main_log.test1()

It works! But then I changed main_log.py to change the debug level. I am used to run "reload()" when updating modules but with this specific module it seems not to be working so it kept showing the same results and that why I was getting always the same results no matter what the debug level was.

>>> reload(main_log) >>> main_log.test1()

Until this point, my only workaround is shutting down Maya completely and re-opening it (guess the same happens within the python interpreter). I will appreciate any further help!

With two files main.py and module.py as shown, I cannot reproduce the behavior you describe; the warning from each is only displayed if logging.basicConfig sets the logging level to logging.WARNING or lower. — chepner
– chepner, Commented May 31, 2014 at 19:53
@chepner you are right, the problem wasn't the code. It was interpreter I was using. Please see my updated question. — chris from local group
– chris from local group, Commented May 31, 2014 at 23:39

chepner · Accepted Answer · 2014-05-31 19:47:36Z

In one sense, yes, basicConfig only configures the root logger. However, any logger you get from logging.getLogger will inherit its configuration from the root logger.

If you want specialized behavior for the loggers in your module, either configure it in your module using methods such as Logger.addHandler and Logger.setLevel, or rely on the main application to use logging.config.fileConfig so that configuration for all loggers can be set via a configuration file. However, in general your module should be concerned with what gets logged, and leave it up to whatever uses the module to determine where, if anywhere, the logged messages are sent.

Addison · Accepted Answer · 2014-05-31 21:38:22Z

Looks like your issue may be because the import module call in main.py ends up running logging.getLogger() in your module.py before your logging.basicConfig() call, which gets a logger that is configured with the default level of WARNING.

Try avoiding calling getLogger() in the module scope.

Note that your getLogger(__name__) calls are returning different logger objects between the different modules, because __name__ is the name of the individual module it's called in.

If you're not writing a multi-threaded program, it may be simpler to just call logging.basicConfig() in your __main__ function, and have all the modules do logging with logging.warning(), logging.error(), etc. Then you can pass in a formatter to your logging.basicConfig() call to show the filename, line number, etc in the log to record what line of code emitted the log.

I was originally evaluating what you said in the last paragraph. However, I wouldn't be able to know from what module the log is coming from right? Or is there a workaround for the root logger to update the name it logs depending on the module that is currently running?
LogRecord object can recognize these special attributes in your format string: docs.python.org/2/library/logging.html#logrecord-attributes, so you can use the module or filename to record where the log came from.

Collectives™ on Stack Overflow

Logging between modules in Python

2 Answers 2

Comments

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Related