Handlers¶
QueueListenerHander¶
A simple QueueHandler
subclass implementation utilizing QueueListener
for configured handlers. This is helpful for detaching the logger handlers from the main threads, which reduces the risk of getting blocked, for example, when using slower handlers such as smtp, file, or socket handlers.
Example Usage¶
An example YAML configuration file utilizing QueueListenerHander
version: 1
objects:
queue:
class: queue.Queue
maxsize: 1000
formatters:
simple:
format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
console:
class: logging.StreamHandler
formatter: simple
stream: ext://sys.stdout
file_handler:
class: logging.FileHandler
filename: test_logger.log
formatter: simple
queue_handler:
class: logging_.handlers.QueueListenerHandler
handlers:
- cfg://handlers.console
- cfg://handlers.file_handler
queue: cfg://objects.queue
loggers:
test_logger:
level: DEBUG
handlers:
- queue_handler
propagate: no
root:
level: NOTSET
handlers:
- console
Just load the configuration file and start logging.
import logging.config
import yaml
with open("logging.yaml", "r") as config_file:
logging.config.dictConfig(yaml.safe_load(config_file.read()))
logger = logging.getLogger("test_logger")
logger.debug("This is a debug log")
logger.info("This is an info log")
logger.warning("This is an warning log")
logger.error("This is an error log")
logger.critical("This is a critical log")
Optional Params¶
A queue object must be passed since the handler does not set a default queue implementation. Set maxsize: -1
to make the queue unlimited.
Module Members¶
- class logging_.handlers.queue_listener_handler.QueueListenerHandler(queue: Any, handlers: Any, respect_handler_level: bool = True, auto_run: bool = True)¶
Bases:
logging.handlers.QueueHandler
QueueListenerHandler class for managing a queue listener with configured handlers.
This class sets up a queue listener logger handler with customizable configurations. Inspired by Rob Blackbourn’s article:
https://rob-blackbourn.medium.com/how-to-use-python-logging-queuehandler-with-dictconfig-1e8b1284e27a
.Example configuration:
# logging.yaml version: 1 objects: queue: class: queue.Queue maxsize: -1 formatters: simple: format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s' handlers: console: class: logging.StreamHandler formatter: simple stream: ext://sys.stdout file_handler: class: logging.FileHandler filename: test_logger.log formatter: simple queue_handler: class: logging_.handlers.QueueListenerHandler handlers: - cfg://handlers.console - cfg://handlers.file_handler queue: cfg://objects.queue
- __init__(queue: Any, handlers: Any, respect_handler_level: bool = True, auto_run: bool = True)¶
Instantiates QueueListenerHandler object.
A simple
QueueHandler
subclass implementation utilizingQueueListener
for configured handlers. This is helpful for detaching your logger handlers from the main processing threads, which reduces the risk of getting blocked, for example, when using slower handlers such as smtp, file, or socket handlers.- Parameters
queue – A queue instance passed from configuration.
handlers – A list of handlers passed from configuration.
respect_handler_level – Flag for overriding logging levels specified in handlers. Default: True.
auto_run – Flag for starting the queue listener automatically. Default: True.
- __module__ = 'logging_.handlers.queue_listener_handler'¶