Python Logging Module: Fundamentals and Advanced Components
Logging is a mechanism for tracking events that occur during the execution of software applications. Developers insert logging calls into their code to signal the occurrence of specific events, which are described with contextual messages that may include variable data unique to each event instance. Events are also categorized by their importance, referred to as a level or severity.
Official documentation: https://docs.python.org/3/library/logging.html
Log Levels
| Level | Use Case |
|---|---|
| DEBUG | Detailed diagnostic information, primarily used for debugging during development |
| INFO | Genarel messages indicating normal application operation |
| WARNING | Indications of unexpected or potentially problematic conditions that don't stop application execution |
| ERROR | Significant issues that prevent part of the application from functioning |
| CRITICAL | Severe errors that render the entire application unable to continue running |
Note: The default logging level is
WARNING.
Advanced Architecture
The logging library follows a modular architecture, consisting of four core components: Loggers, Handlers, Filters, and Formatters.
Loggers
Loggers provide the primary interface that application code interacts with. Key operations include:
- Retrieving a logger instance (singleton pattern):
import logging # Get a logger instance tied to the current module app_logger = logging.getLogger(__name__) - Setting the minimum severity level for events the logger will process:
# Set logger to capture DEBUG-level and above events app_logger.setLevel(logging.DEBUG) - Attaching or detaching handlers to route events to different outputs:
# Example: Add a handler to the logger app_logger.addHandler(console_handler) # Example: Remove a handler from the logger app_logger.removeHandler(console_handler)
Handlers
Handlers route logged events to specific destinations such as console output, files, email, or network services via protocols like HTTP or Socket. Common handler types include:
StreamHandler: Sends log output to standard streams (e.g., console)FileHandler: Writes log events to a file on disk
Example implementations:
# Create a StreamHandler for console output
console_handler = logging.StreamHandler()
# Create a FileHandler to append logs to a file with UTF-8 encoding
file_handler = logging.FileHandler(
filename="app_activity.log",
mode="a",
encoding="utf-8",
delay=False
)
Filters
Filters provide granular control over wich log events are processed. They can be attached to Loggers or Handlers to filter events based on custom criteria, such as allowing only logs from a specific module or containing certain keywords.
Formatters
Formatters define the structure and content of the final log output. They control the order of feilds like timestamp, log level, message, and module name.
The constructor for a Formatter accepts parameters to customize the message format, timestamp format, and string style:
# Create a formatter with custom message and timestamp format
log_formatter = logging.Formatter(
fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
style="%"
)
The style parameter defaults to %, which uses the classic %(key)s string formatting syntax. The default timestamp format is %Y-%m-%d %H:%M:%S if datefmt is not specified.