Elegant way to make logging.LoggerAdapter available to other modules

Question

I use a LoggerAdapter to let my python logging output Linux TIDs instead of the long unique IDs. But this way I don't modify an existing logger but I create a new object:

    new_logger = logging.LoggerAdapter(
                    logger=logging.getLogger('mylogger'), 
                    extra=my_tid_extractor())

Now I want this LoggerAdapter be used by certain modules. As long as I know a global variable being used as logger I can do something like this:

    somemodule.logger = new_logger

But this is not nice - it works only in a couple of cases and you need to know the logger variables used by the modules.

Do you know a way to make a LoggerAdapter available globally e.g. by calling s.th. like

    logging.setLogger('mylogger', new_logger)

Or alternatively: is there some other way to let Python logging output Linux thread IDs like printed by ps?

Dmitry Nedbaylo · Accepted Answer · 2015-01-20T17:20:36.333

Alternatively, you can to implement custom logger, and make it default in logging module.

Here is example:

import logging
import ctypes

SYS_gettid = 186
libc = ctypes.cdll.LoadLibrary('libc.so.6')

FORMAT = '%(asctime)-15s [thread=%(tid)s] %(message)s'
logging.basicConfig(level=logging.DEBUG, format=FORMAT)

def my_tid_extractor():
    tid = libc.syscall(SYS_gettid)
    return {'tid': tid}

class CustomLogger(logging.Logger):

    def _log(self, level, msg, args, exc_info=None, extra=None):
        if extra is None:
            extra = my_tid_extractor()
        super(CustomLogger, self)._log(level, msg, args, exc_info, extra)

logging.setLoggerClass(CustomLogger)


logger = logging.getLogger('test')
logger.debug('test')

Output sample:

2015-01-20 19:24:09,782 [thread=5017] test

score 3 · Answer 2 · answered Dec 28 '15 at 11:46

I think you need override LoggerAdapter.process() method Because the default LoggerAdapter.process method will do nothing, Here is example：

import logging
import random
L=logging.getLogger('name')

class myLogger(logging.LoggerAdapter):
    def process(self,msg,kwargs):
        return '(%d),%s' % (self.extra['name1'](1,1000),msg)  ,kwargs

#put the randint function object  
LA=myLogger(L,{'name1':random.randint})

#now,do some logging
LA.debug('some_loging_messsage')

out>>DEBUG:name:(167),some_loging_messsage

score 3 · Answer 3 · answered Jul 07 '21 at 13:59

I had a similar problem. My solution might be a bit more generic than the accepted one.

I’ve also used a custom logger class, but I did a generic extension that allows me to register adapters after it’s instantiated.

class AdaptedLogger(logging.Logger):
    """A logger that allows you to register adapters on a instance."""

    def __init__(self, name):
        """Create a new logger instance."""
        super().__init__(name)
        self.adapters = []

    def _log(self, level, msg, *args, **kwargs):
        """Let adapters modify the message and keyword arguments."""
        for adapter in self.adapters:
            msg, kwargs = adapter.process(msg, kwargs)
        return super()._log(level, msg, *args, **kwargs)

To make you logger use the class you have to instantiate it before it is used elsewhere. For example using:

original_class = logging.getLoggerClass()
logging.setLoggerClass(AdaptedLogger)
logcrm_logger = logging.getLogger("test")
logging.setLoggerClass(original_class)

Then you can register adapters on the instance at any time later on.

logger = logging.getLogger("test")
adapter = logging.LoggerAdapter(logger, extra=my_tid_extractor())
logger.adapters.append(adapter)

Actually the “adapters” can be any object now as long as they have a process-method with a signature compatible with logging.LoggingAdapter.process().

Elegant way to make logging.LoggerAdapter available to other modules

3 Answers3