Logging works locally but not on remote

Question

I set up a basic python logger that writes to a log file and to stdout. When I run my python program locally, log messages with logging.info appear as expected in the file and in the console. However, when I run the same program remotely via ssh -n user@server python main.py neither the console nor the file show any logging.info messages.

This is the code used to set up the logger:

def setup_logging(model_type, dataset): file_name = dataset + "_" + model_type + time.strftime("_%Y%m%d_%H%M") logging.basicConfig( level=logging.INFO, format="[%(levelname)-5.5s %(asctime)s] %(message)s", datefmt='%H:%M:%S', handlers=[ logging.FileHandler("log/{0}.log".format(file_name)), logging.StreamHandler() ])

I already tried the following things:

Sending a message to logging.warning: Those appear as expected on the root logger. However, even without setting up the logger and falling back to the default logging.info messages do not show up.
The file and folder permissions seem to be alright and an empty file is created on disk.
Using print works as usual as well

newtover · Accepted Answer · 2019-05-06 15:25:23Z

If you look into the source code of basicConfig function, you will see that the function is applied only when there are no handlers on the root logger:

_acquireLock() try: force = kwargs.pop('force', False) if force: for h in root.handlers[:]: root.removeHandler(h) h.close() if len(root.handlers) == 0: handlers = kwargs.pop("handlers", None) if handlers is None: ...

I think, one of the libraries you use configures logging on import. And as you see from the sample above, one of the solutions is to use force=True argument.

A possible disadvantage is that several popular data-science libraries keep a reference to the loggers they configure, so that when you reconfigure logging yourselves their old loggers with the handlers are still there and do not see your changes. In which case you will also need to clean the handlers for those loggers as well.

The force option seems to be added only in python 3.8 so I unfortunately cannot try that (running 3.7), but I am not sure I understand why this problem would then only appear when I execute the code remotely? EDIT: You were indeed right. I followed this to reload the logger and now it is working on the remote as well.
@smonsays, the possible reason why the code works differently remotely is the environment. It was my first guess, because I saw this in practice several times. I'm glad this helped.

Collectives™ on Stack Overflow

Logging works locally but not on remote

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related