Always Reference Logger Object in Main-CodePudding

Is it possible for a library of code to obtain a reference to a logger object created by client code that uses a unique name?

The python advanced logging tutorial says the following:

A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows: logger = logging.getLogger(__name__) This means that logger names track the package/module hierarchy, and it’s intuitively obvious where events are logged just from the logger name.

Every module in my library does:

LOG = logging.getLogger(__name__)

This works fine when client code does something like:

logger = logging.getLogger()

But breaks (I get no log output to the registered handlers from the logging object in main) when client code does something like:

logger = logging.getLogger('some.unique.path')

Because I am packaging my code as a library to be used with a lot of different clients, I want to have the most extensible logging. That is, I want my module level logging to reference the same logger object as main whenever possible, whether the client code is using a named logger or not.

Here is an example program to reproduce on your end. Imagine test.py is my library of code that I want to always reference any logger created in main.py.

Example Output

% python3 main.py
in main
%

Desired Output

% python3 main.py
in main
hi
%

main.py

import logging
from test import somefunc

LOG = logging.getLogger('some.unique.path')
LOG.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
LOG.addHandler(ch)

def main():
    LOG.info('in main')
    somefunc()

if __name__ == '__main__':
    main()

test.py

import logging

LOG = logging.getLogger(__name__)

def somefunc():
    LOG.info('hi')

CodePudding user response：

This is the approach I would take.

Create a separate logging utility. I have attached the code for it below.
Now, import this logger utility (or) class (or) function that this provides, wherever needed.

Updated your code, assuming I am using the above linked logger utility and tested it locally on my machine.

Code for main.py

from logger import get_logger
from test import somefunc

LOG = get_logger()

def main():
    LOG.info("in main")
    somefunc()


if __name__ == "__main__":
    main()

Code for test.py

from logger import get_logger

LOG = get_logger()

def somefunc():
    LOG.info("hi")

Code for logger.py - Attaching the code here too

import logging
from logging.handlers import RotatingFileHandler


bytes_type = bytes
unicode_type = str
basestring_type = str

DEFAULT_LOGGER = "default"
INTERNAL_LOGGER_ATTR = "internal"
CUSTOM_LOGLEVEL = "customloglevel"
logger = None

_loglevel = logging.DEBUG
_logfile = None
_formatter = None


def get_logger(
        name=None,
        logfile=None,
        level=logging.DEBUG,
        maxBytes=0,
        backupCount=0,
        fileLoglevel=None,
):
    _logger = logging.getLogger(name or __name__)
    _logger.propagate = False
    _logger.setLevel(level)


    # Reconfigure existing handlers
    has_stream_handler = False
    for handler in list(_logger.handlers):
        if isinstance(handler, logging.StreamHandler):
            has_stream_handler = True

        if isinstance(handler, logging.FileHandler) and hasattr(
                handler, INTERNAL_LOGGER_ATTR
        ):
            # Internal FileHandler needs to be removed and re-setup to be able
            # to set a new logfile.
            _logger.removeHandler(handler)
            continue

        # reconfigure handler
        handler.setLevel(level)

    if not has_stream_handler:
        stream_handler = logging.StreamHandler()
        setattr(stream_handler, INTERNAL_LOGGER_ATTR, True)
        stream_handler.setLevel(level)
        _logger.addHandler(stream_handler)

    if logfile:
        rotating_filehandler = RotatingFileHandler(
            filename=logfile, maxBytes=maxBytes, backupCount=backupCount
        )
        setattr(rotating_filehandler, INTERNAL_LOGGER_ATTR, True)
        rotating_filehandler.setLevel(fileLoglevel or level)
        _logger.addHandler(rotating_filehandler)

    return _logger


def setup_default_logger(
        logfile=None, level=logging.DEBUG, formatter=None, maxBytes=0, backupCount=0
):
    global logger
    logger = get_logger(
        name=DEFAULT_LOGGER, logfile=logfile, level=level, formatter=formatter
    )
    return logger


def reset_default_logger():
    """
    Resets the internal default logger to the initial configuration
    """
    global logger
    global _loglevel
    global _logfile
    global _formatter
    _loglevel = logging.DEBUG
    _logfile = None
    _formatter = None
    logger = get_logger(name=DEFAULT_LOGGER, logfile=_logfile, level=_loglevel)


# Initially setup the default logger
reset_default_logger()


def loglevel(level=logging.DEBUG, update_custom_handlers=False):
    """
    Set the minimum loglevel for the default logger
    Reconfigures only the internal handlers of the default logger (eg. stream and logfile).
    Update the loglevel for custom handlers by using `update_custom_handlers=True`.

    :param int level: Minimum logging-level (default: `logging.DEBUG`).
    :param bool update_custom_handlers: custom handlers to this logger; set `update_custom_handlers` to `True`
    """
    logger.setLevel(level)

    # Reconfigure existing internal handlers
    for handler in list(logger.handlers):
        if hasattr(handler, INTERNAL_LOGGER_ATTR) or update_custom_handlers:
            # Don't update the loglevel if this handler uses a custom one
            if hasattr(handler, CUSTOM_LOGLEVEL):
                continue

            # Update the loglevel for all default handlers
            handler.setLevel(level)

    global _loglevel
    _loglevel = level


def formatter(formatter, update_custom_handlers=False):
    """
    Set the formatter for all handlers of the default logger
    :param Formatter formatter: default uses the internal LogFormatter.
    :param bool update_custom_handlers: custom handlers to this logger - set ``update_custom_handlers`` to `True`
    """
    for handler in list(logger.handlers):
        if hasattr(handler, INTERNAL_LOGGER_ATTR) or update_custom_handlers:
            handler.setFormatter(formatter)

    global _formatter
    _formatter = formatter


def logfile(
        filename,
        formatter=None,
        mode="a",
        maxBytes=0,
        backupCount=0,
        encoding=None,
        loglevel=None,
):
    """
    Function to handle the rotating fileHandler
    :param filename: filename logs are being collected
    :param mode: fileMode
    :param maxBytes: values for roll-over at a pre-determined size; if zero; rollover never occurs
    :param backupCount: if value is non-zero; system saves old logfiles; by appending extensions
    :param encoding: set encoding option; if not None; open file with that encoding.
    :param loglevel: loglevel set
    """
    # Step 1: If an internal RotatingFileHandler already exists, remove it
    for handler in list(logger.handlers):
        if isinstance(handler, RotatingFileHandler) and hasattr(
                handler, INTERNAL_LOGGER_ATTR
        ):
            logger.removeHandler(handler)

    # Step 2: If wanted, add the RotatingFileHandler now
    if filename:
        rotating_filehandler = RotatingFileHandler(
            filename,
            mode=mode,
            maxBytes=maxBytes,
            backupCount=backupCount,
            encoding=encoding,
        )

        # Set internal attributes on this handler
        setattr(rotating_filehandler, INTERNAL_LOGGER_ATTR, True)
        if loglevel:
            setattr(rotating_filehandler, CUSTOM_LOGLEVEL, True)

        # Configure the handler and add it to the logger
        rotating_filehandler.setLevel(loglevel or _loglevel)
        logger.addHandler(rotating_filehandler)

Output:

in main
hi

Note: Do dive deep into the Logger Utility linked to understand all the internal details.

CodePudding user response：

Use logging.basicConfig instead of manually trying to configure a logger. Loggers inherit their configuration from their parent.

import logging
from test import somefunc

LOG = logging.getLogger('some.unique.path')

def main():
    LOG.info('in main')
    somefunc()

if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    main()