Handlers¶
QueueListenerHander¶
A simple QueueHandler
subclass implementation utilizing QueueListener
for configured handlers. This is helpful for detaching the logger handlers from the main threads, which reduces the risk of getting blocked, for example, when using slower handlers such as smtp, file, or socket handlers.
An example YAML configuration file utilizing QueueListenerHander
version: 1
objects:
queue:
class: queue.Queue
maxsize: 1000
formatters:
simple:
format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
console:
class: logging.StreamHandler
formatter: simple
stream: ext://sys.stdout
file_handler:
class: logging.FileHandler
filename: 'test_logger.log'
formatter: simple
queue_handler:
class: logging_.handlers.QueueListenerHandler
handlers:
- cfg://handlers.console
- cfg://handlers.file_handler
queue: cfg://objects.queue
loggers:
test_logger:
level: DEBUG
handlers:
- queue_handler
propagate: no
root:
level: NOTSET
handlers:
- console
Note: A queue object must be passed since the handler does not set a default queue implementation. Set maxsize: -1
to make the queue unlimited.
Just load the configuration file and start logging.
import logging.config
import yaml
with open("logging.yaml", "r") as config_file:
logging.config.dictConfig(yaml.safe_load(config_file.read()))
logger = logging.getLogger("test_logger")
logger.debug("This is a debug log")
logger.info("This is an info log")
logger.warning("This is an warning log")
logger.error("This is an error log")
logger.critical("This is a critical log")
Module Members¶
- class logging_.handlers.queue_listener_handler.QueueListenerHandler(queue: Any, handlers: Any, respect_handler_level: bool = True, auto_run: bool = True)¶
Bases:
logging.handlers.QueueHandler
QueueListenerHandler class for managing a queue listener with configured handlers.
This class sets up a queue listener logger handler with customizable configurations. Inspired by Rob Blackbourn’s [article](https://rob-blackbourn.medium.com/how-to-use-python-logging-queuehandler-with-dictconfig-1e8b1284e27a).
Example configuration:
# logging.yaml objects: queue: class: queue.Queue maxsize: -1 formatters: simple: format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s' handlers: console: class: logging.StreamHandler formatter: simple stream: ext://sys.stdout file_handler: class: logging.FileHandler filename: 'config_test.log' formatter: simple queue_handler: class: logging_.handlers.QueueListenerHandler handlers: - cfg://handlers.console - cfg://handlers.file_handler queue: cfg://objects.queue
- __init__(queue: Any, handlers: Any, respect_handler_level: bool = True, auto_run: bool = True)¶
Instantiates QueueListenerHandler object.
A simple QueueHandler subclass implementation utilizing QueueListener for configured handlers. This is helpful for detaching ypur logger handlers from the main processing threads, which reduces the risk of getting blocked, for example, when using slower handlers such as smtp, file, or socket handlers.
- Parameters
queue – A queue instance passed from configuration.
handlers – A list of handlers passed from configuration.
respect_handler_level (bool) – Flag for overriding logging levels specified in handlers. Default: True.
auto_run (bool) – Flag for starting the queue listener automatically. Default: True.
- __module__ = 'logging_.handlers.queue_listener_handler'¶
- emit(record: logging.LogRecord)¶
Processes the specified logging record.
This method is implemented to avoid raising a NotImplementedError from the subclass.
- Parameters
record (LogRecord) – A logging.LogRecord object.