how to maintain separate log files for each specific log level in python?-CodePudding

when i set level to INFO in file_handler. am getting other log levels also printed into the file. how can i get each log level printed into different log file . i dont want duplicate logs in any of the files.Can any one please help?

import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s : %(name)s :%(levelname)s :%(message)s')
formatter1 = logging.Formatter('%(levelname)s :%(message)s')
file_handler = logging.FileHandler('test_log.log')
file_handler.setLevel(logging.INFO) 
file_handler.setFormatter(formatter)
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.WARNING)
stream_handler.setFormatter(formatter1)
logger.addHandler(file_handler)
logger.addHandler(stream_handler)
logger.error("this is error")
logger.debug("this is debug")
logger.info("this is info")
logger.critical("this is critical")
logger.warning("this is warning")

i tried this and am getting all the other log levels also into the log file

CodePudding user response：

All the levels which you have logged at are above or at the set level for your handler (INFO), so those log messages will all appear in your log file.

You can write custom filters that will allow you to include ONLY certain levels and exclude all levels above and below.

Filters can be set on a handler level to allow for all levels to have their own file.

A word of warning: the reason this is not so easy to do is because it goes against convention. Omitting warning, error, and critical logs from an info log file could confuse anyone looking at those logs into thinking that everything is working properly.

It might be a worthwhile exersize to think about the underlying reason why you want them separated and brainstorm a way to achieve your goals without going against convention

CodePudding user response：

Logging level determines the minimum logging level. For your case, Filters can be used to filter only pass the desired level.

This is your example code with the required change. You can add any filtering logic to LoggingLevelFilter if you want for example to log several logging levels.

import logging


class LoggingLevelFilter(logging.Filter):
    def __init__(self, logging_level: int):
        super().__init__()
        self.logging_level = logging_level

    def filter(self, record: logging.LogRecord) -> bool:
        return record.levelno == self.logging_level


logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s : %(name)s :%(levelname)s :%(message)s')
formatter1 = logging.Formatter('%(levelname)s :%(message)s')

file_handler = logging.FileHandler('test_log1.log')
file_handler.setLevel(logging.INFO)
file_handler.setFormatter(formatter)
file_handler.addFilter(LoggingLevelFilter(logging.INFO))

stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.WARNING)
stream_handler.setFormatter(formatter1)
stream_handler.addFilter(LoggingLevelFilter(logging.WARNING))

logger.addHandler(file_handler)
logger.addHandler(stream_handler)

logger.error("this is error")
logger.debug("this is debug")
logger.info("this is info")
logger.critical("this is critical")
logger.warning("this is warning")