Skip to content

Add response auditing capability to Pinot's audit logging system#2

Open
suvodeep-pyne wants to merge 5 commits intomasterfrom
auditresponse
Open

Add response auditing capability to Pinot's audit logging system#2
suvodeep-pyne wants to merge 5 commits intomasterfrom
auditresponse

Conversation

@suvodeep-pyne
Copy link
Owner

Summary

This PR enhances Pinot's audit logging system by adding response auditing capability, enabling complete request-response cycle tracking for better observability and compliance.

Changes

  • AuditLogFilter: Now implements both ContainerRequestFilter and ContainerResponseFilter interfaces
  • Request-Response Correlation: Uses UUID-based request IDs to correlate request and response audit events
  • Performance Metrics: Captures API execution duration using high-precision timing (System.nanoTime())
  • Conditional Processing: Response context is only created when response auditing is enabled
  • JSON Serialization: Fixed duplicate field issues by using JsonAutoDetect annotations

Key Features

  1. Response Auditing: Captures HTTP response code and execution duration
  2. Request ID Correlation: Each request gets a unique UUID for tracking through the system
  3. Configurable: Response auditing can be enabled/disabled via capture.response.enabled config
  4. Performance Optimized: Context objects only created when needed
  5. Graceful Degradation: Audit failures never affect main request processing

Files Modified

  • AuditConfig.java: Added capture.response.enabled configuration field
  • AuditEvent.java: Added request_id, response_code, and duration_ms fields with proper JSON annotations
  • AuditLogFilter.java: Implemented response filter with timing and correlation logic
  • AuditResponseContext.java: New context wrapper for passing data between filters
  • AuditRequestProcessor.java: Minor refactoring for consistency

Configuration

To enable response auditing:

pinot.audit.controller.enabled=true
pinot.audit.controller.capture.response.enabled=true

Sample Output

Request audit log:

{
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "timestamp": "2025-01-17T10:30:45.123Z",
  "service_id": "controller",
  "endpoint": "/tables/myTable",
  "method": "POST",
  "user_id": {"principal": "user@company.com"}
}

Response audit log:

{
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "timestamp": "2025-01-17T10:30:45.456Z",
  "endpoint": "/tables/myTable",
  "method": "POST",
  "response_code": 200,
  "duration_ms": 333
}

Testing

  • Compiled successfully with Maven
  • Response auditing only activates when explicitly enabled
  • JSON serialization produces clean output without duplicate fields

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant