Not able to setup python logger properly

Question

Im trying to setup logging for my module:

Used the exact snippet hosted here: https://gist.github.com/kingspp/9451566a5555fb022215ca2b7b802f19

logging.yaml:

 info_file_handler: class: logging.handlers.RotatingFileHandler level: INFO formatter: standard filename: /tmp/info.log maxBytes: 10485760 # 10MB backupCount: 20 encoding: utf8

setup_logging.py

 def setup_logging(default_path='logging.yaml', default_level=logging.INFO, env_key='LOG_CFG'): path = default_path value = os.getenv(env_key, None) if value: path = value if os.path.exists(path): with open(path, 'rt') as f: try: config = yaml.safe_load(f.read()) logging.config.dictConfig(config)

Im initiating this in server.py as: moto/server.py

 setup_logging() logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) logger.info("Logger SETUP successfully")

This logs the output to the /tmp/info.log appropriately but when I initialize the same in another file in a child module Im not seeing the logs getting written to the file:

moto/ec2/models.py

 logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) def describe_internet_gateways( self, internet_gateway_ids=None, filters=None): igws = [] #logger = logging.getLogger(__name__) #logger.setLevel(logging.DEBUG) print("\n\n\n\n\n{}\n\n\n\n\n".format(__name__)) logger.info("in describe_internet_gateways") session = get_db_session() igws = session.query(InternetGateway_t)

But if I initialize the logger inside the method(if I uncomment the commented section inside the method above) then the logs are written appropriately in the /tmp/info.log file

Could someone please help me what Im missing out, how to initialize the logs only once and use it across modules?

Read and tried the logging module explanation from python docs, but Im stilling hitting the issue.

Edit-1: Tried the solution below by @olinox14 below by doing the following edits:

logging.json

 "handlers": { "console": { "class": "logging.StreamHandler", "level": "DEBUG", "formatter": "standard", "stream": "ext://sys.stdout" }, "info_file_handler": { "class": "logging.handlers.RotatingFileHandler", "level": "INFO", "formatter": "standard", "filename": "/tmp/info.log", "maxBytes": 10485760, "backupCount": 20, "encoding": "utf8" }, "error_file_handler": { "class": "logging.handlers.RotatingFileHandler", "level": "ERROR", "formatter": "error", "filename": "/tmp/errors.log", "maxBytes": 10485760, "backupCount": 20, "encoding": "utf8" }, "debug_file_handler": { "class": "logging.handlers.RotatingFileHandler", "level": "DEBUG", "formatter": "standard", "filename": "/tmp/debug.log", "maxBytes": 10485760, "backupCount": 20, "encoding": "utf8" }, "critical_file_handler": { "class": "logging.handlers.RotatingFileHandler", "level": "CRITICAL", "formatter": "standard", "filename": "/tmp/critical.log", "maxBytes": 10485760, "backupCount": 20, "encoding": "utf8" }, "warn_file_handler": { "class": "logging.handlers.RotatingFileHandler", "level": "WARN", "formatter": "standard", "filename": "/tmp/warn.log", "maxBytes": 10485760, "backupCount": 20, "encoding": "utf8" } }, "root": { "level": "NOTSET", "handlers": [ "console", ], "propagate": true }, "loggers": { "test": { "level": "DEBUG", "handlers": [ "console", "info_file_handler", "error_file_handler", "critical_file_handler", "debug_file_handler", "warn_file_handler" ], "propagate": "no" } } }

But now Im seeing logs on console but not logged in the file.

olinox14 · Accepted Answer · 2019-09-11 10:14:32Z

Basically, named loggers are singletons. You setup a new logger each time you call a new name, and you retrieve the existing one if you call the reviously used name.

Here is a solution:

logging.yaml:

version: 1 formatters: standard: format: "%(asctime)s - %(levelname)s - %(message)s" handlers: info_file_handler: class: logging.handlers.RotatingFileHandler level: INFO formatter: standard filename: info.log maxBytes: 10485760 # 10MB backupCount: 20 encoding: utf8 loggers: my_logger: level: DEBUG handlers: [info_file_handler] propagate: no

moto/server.py

import logging.config import os import yaml from ec2 import models def setup_logging(default_path='logging.yaml', default_level=logging.INFO, env_key='LOG_CFG'): path = os.getenv(env_key, None) or default_path try: with open(path, 'rt') as f: config = yaml.safe_load(f.read()) logging.config.dictConfig(config) except FileNotFoundError: #... load the basic config here? pass setup_logging() logger = logging.getLogger("my_logger") logger.info("Logger SETUP successfully") models.f()

moto/ec2/models.py

import logging logger = logging.getLogger("my_logger") def f(): logger.info("Logger called successfully")

I cannot help you if I don't have the config of the handlers, since they are responsible of outputting the logging messages into a file, the console, a database and so on... The handler above does write messages to ` info.log` file
You called the logger 'test' here, did you replace every logger = logging.getLogger(__name__) by logger = logging.getLogger("test") already?
That's why i asked: it seems the 'root' logger is used, not the 'test' one. Or maybe tha handlers are not configured like they should, try to add the 'info_file_handler' to the root logger maybe? also does /tmp/ exists? Do you have right to write in it? Maybe try to remove the /tmp/ part to let the file being created in the working directory?
Go the issue, had to set propagate in loggers to false instead of "no", since it is json. Thanks for all the help olinox14

Collectives™ on Stack Overflow

Not able to setup python logger properly

1 Answer 1

11 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

11 Comments

Related