170 lines
6.6 KiB
Markdown
170 lines
6.6 KiB
Markdown
# Structured Logging Implementation
|
|
|
|
## Problem Solved
|
|
Basic print() statements throughout the codebase made debugging difficult in production environments, with no request tracking, structured data, or proper log management.
|
|
|
|
## Solution Implemented
|
|
|
|
### 1. **Comprehensive Logging Infrastructure** (`session-manager/logging_config.py`)
|
|
- **Structured JSON Formatter**: Machine-readable logs for production analysis
|
|
- **Human-Readable Formatter**: Clear logs for development and debugging
|
|
- **Request Context Tracking**: Automatic request ID propagation across operations
|
|
- **Logger Adapter**: Request-aware logging with thread-local context
|
|
- **Performance Logging**: Built-in metrics for operations and requests
|
|
- **Security Event Logging**: Dedicated audit trail for security events
|
|
|
|
### 2. **Application Integration** (`session-manager/main.py`)
|
|
- **FastAPI Integration**: Request context automatically set for all endpoints
|
|
- **Performance Tracking**: Request timing and session operation metrics
|
|
- **Error Logging**: Structured error reporting with context
|
|
- **Security Auditing**: Authentication and proxy access logging
|
|
- **Lifecycle Logging**: Application startup/shutdown events
|
|
|
|
### 3. **Production-Ready Features**
|
|
- **Log Rotation**: Automatic file rotation with size limits and backup counts
|
|
- **Environment Detection**: Auto-detection of development vs production environments
|
|
- **Third-Party Integration**: Proper log level configuration for dependencies
|
|
- **Resource Management**: Efficient logging with minimal performance impact
|
|
- **Filtering and Aggregation**: Support for log aggregation systems
|
|
|
|
### 4. **Testing & Validation Suite**
|
|
- **Formatter Testing**: JSON and human-readable format validation
|
|
- **Context Management**: Request ID tracking and thread safety
|
|
- **Log Level Filtering**: Proper level handling and filtering
|
|
- **Structured Data**: Extra field inclusion and JSON validation
|
|
- **Environment Configuration**: Dynamic configuration from environment variables
|
|
|
|
## Key Technical Improvements
|
|
|
|
### Before (Basic Print Statements)
|
|
```python
|
|
print("Starting Session Management Service")
|
|
print(f"Container {container_name} started on port {port}")
|
|
print(f"Warning: Could not load sessions file: {e}")
|
|
# No request tracking, no structured data, no log levels
|
|
```
|
|
|
|
### After (Structured Logging)
|
|
```python
|
|
with RequestContext():
|
|
logger.info("Starting Session Management Service")
|
|
log_session_operation(session_id, "container_started", port=port)
|
|
log_security_event("authentication_success", "info", session_id=session_id)
|
|
```
|
|
|
|
### JSON Log Output (Production)
|
|
```json
|
|
{
|
|
"timestamp": "2024-01-15T10:30:45.123Z",
|
|
"level": "INFO",
|
|
"logger": "session_manager.main",
|
|
"message": "Container started successfully",
|
|
"request_id": "req-abc123",
|
|
"session_id": "ses-xyz789",
|
|
"operation": "container_start",
|
|
"port": 8081,
|
|
"duration_ms": 1250.45
|
|
}
|
|
```
|
|
|
|
### Request Tracing Across Operations
|
|
```python
|
|
@app.post("/sessions")
|
|
async def create_session(request: Request):
|
|
with RequestContext(): # Automatic request ID generation
|
|
# All logs in this context include request_id
|
|
log_session_operation(session_id, "created")
|
|
# Proxy requests also include same request_id
|
|
# Cleanup operations maintain request context
|
|
```
|
|
|
|
## Implementation Details
|
|
|
|
### Log Formatters
|
|
- **JSON Formatter**: Structured data with timestamps, levels, and context
|
|
- **Human Formatter**: Developer-friendly with request IDs and readable timestamps
|
|
- **Extra Fields**: Automatic inclusion of request_id, session_id, user_id, etc.
|
|
|
|
### Request Context Management
|
|
- **Thread-Local Storage**: Request IDs isolated per thread/async task
|
|
- **Context Managers**: Automatic cleanup and nesting support
|
|
- **Global Access**: RequestContext.get_current_request_id() anywhere in call stack
|
|
|
|
### Performance & Security Logging
|
|
```python
|
|
# Performance tracking
|
|
log_performance("create_session", 245.67, session_id=session_id)
|
|
|
|
# Request logging
|
|
log_request("POST", "/sessions", 200, 245.67, session_id=session_id)
|
|
|
|
# Security events
|
|
log_security_event("authentication_failure", "warning",
|
|
session_id=session_id, ip_address="192.168.1.1")
|
|
```
|
|
|
|
### Configuration Management
|
|
```python
|
|
# Environment-based configuration
|
|
LOG_LEVEL=INFO # Log verbosity
|
|
LOG_FORMAT=auto # json/human/auto
|
|
LOG_FILE=/var/log/app.log # File output
|
|
LOG_MAX_SIZE_MB=10 # Rotation size
|
|
LOG_BACKUP_COUNT=5 # Backup files
|
|
```
|
|
|
|
## Production Deployment
|
|
|
|
### Log Aggregation Integration
|
|
Structured JSON logs integrate seamlessly with:
|
|
- **ELK Stack**: Elasticsearch, Logstash, Kibana
|
|
- **Splunk**: Enterprise log aggregation
|
|
- **CloudWatch**: AWS log management
|
|
- **DataDog**: Observability platform
|
|
- **Custom Systems**: JSON parsing for any log aggregation tool
|
|
|
|
### Monitoring & Alerting
|
|
- **Request Performance**: Track API response times and error rates
|
|
- **Security Events**: Monitor authentication failures and suspicious activity
|
|
- **System Health**: Application startup, errors, and resource usage
|
|
- **Business Metrics**: Session creation, proxy requests, cleanup operations
|
|
|
|
### Log Analysis Queries
|
|
```sql
|
|
-- Request performance analysis
|
|
SELECT request_id, AVG(duration_ms) as avg_duration
|
|
FROM logs WHERE operation = 'http_request'
|
|
GROUP BY DATE(timestamp)
|
|
|
|
-- Security event monitoring
|
|
SELECT COUNT(*) as auth_failures
|
|
FROM logs WHERE security_event = 'authentication_failure'
|
|
AND timestamp > NOW() - INTERVAL 1 HOUR
|
|
|
|
-- Session lifecycle tracking
|
|
SELECT session_id, COUNT(*) as operations
|
|
FROM logs WHERE session_id IS NOT NULL
|
|
GROUP BY session_id
|
|
```
|
|
|
|
## Validation Results
|
|
|
|
### Logging Functionality ✅
|
|
- **Formatters**: JSON and human-readable formats working correctly
|
|
- **Request Context**: Thread-local request ID tracking functional
|
|
- **Log Levels**: Proper filtering and level handling
|
|
- **Structured Data**: Extra fields included in JSON output
|
|
|
|
### Application Integration ✅
|
|
- **FastAPI Endpoints**: Request context automatically applied
|
|
- **Performance Metrics**: Request timing and operation tracking
|
|
- **Error Handling**: Structured error reporting with context
|
|
- **Security Logging**: Authentication and access events captured
|
|
|
|
### Production Readiness ✅
|
|
- **Log Rotation**: File size limits and backup count working
|
|
- **Environment Detection**: Auto-selection of appropriate format
|
|
- **Resource Efficiency**: Minimal performance impact on application
|
|
- **Scalability**: Works with high-volume logging scenarios
|
|
|
|
The structured logging system transforms basic print statements into a comprehensive observability platform, enabling effective debugging, monitoring, and operational visibility in both development and production environments. 🔍 |