- 
          
- 
                Notifications
    You must be signed in to change notification settings 
- Fork 7k
Add Observability page to docs #9723
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Open
      
      
            itssimon
  wants to merge
  7
  commits into
  encode:main
  
    
      
        
          
  
    
      Choose a base branch
      
     
    
      
        
      
      
        
          
          
        
        
          
            
              
              
              
  
           
        
        
          
            
              
              
           
        
       
     
  
        
          
            
          
            
          
        
       
    
      
from
itssimon:docs-topics-observability
  
      
      
   
  
    
  
  
  
 
  
      
    base: main
Could not load branches
            
              
  
    Branch not found: {{ refName }}
  
            
                
      Loading
              
            Could not load tags
            
            
              Nothing to show
            
              
  
            
                
      Loading
              
            Are you sure you want to change the base?
            Some commits from the old base branch may be removed from the timeline,
            and old review comments may become outdated.
          
          
  
     Open
                    Changes from all commits
      Commits
    
    
            Show all changes
          
          
            7 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      b3f6138
              
                First draft
              
              
                itssimon 5288b05
              
                Tweaks
              
              
                itssimon 4e073c1
              
                Fix formatting
              
              
                itssimon 45cfc66
              
                Fix formatting again
              
              
                itssimon 8ca33c6
              
                Move Prometheus section below OpenTelemetry
              
              
                itssimon 04a0871
              
                Move Observability to bottom of Topics menu
              
              
                itssimon 6574c6c
              
                Remove django-silk
              
              
                itssimon File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Observability | ||
|  | ||
| > Measurement is the first step that leads to control and eventually to improvement. If you can't measure something, you can't understand it. If you can't understand it, you can't control it. If you can't control it, you can't improve it. | ||
| > | ||
| > — H. James Harrington | ||
|  | ||
| REST framework doesn’t include built-in observability features, but it works well with popular tools, standards, and third party packages. For simpler needs, you can also implement custom logging or metrics collection using Django middleware. | ||
|  | ||
| ## Custom request logging | ||
|  | ||
| You can implement a custom middleware that logs relevant information about handled API requests and responses using Python's built-in `logging` module. | ||
|  | ||
| ```python | ||
| import logging | ||
| import time | ||
|  | ||
| logger = logging.getLogger("your_app.requests") | ||
|  | ||
|  | ||
| def request_logging_middleware(get_response): | ||
| def middleware(request): | ||
| start_time = time.time() | ||
| response = get_response(request) | ||
| duration = time.time() - start_time | ||
| logger.info( | ||
| f"{request.method} {request.path} - {response.status_code} {response.reason_phrase} - {int(duration*1000)}ms" | ||
| ) | ||
| return response | ||
|  | ||
| return middleware | ||
| ``` | ||
|  | ||
| Then, add the middleware to your Django settings. | ||
|  | ||
| ```python | ||
| MIDDLEWARE = [ | ||
| "your_app.middleware.request_logging_middleware", | ||
| # ... other middleware | ||
| ] | ||
| ``` | ||
|  | ||
| ## OpenTelemetry | ||
|  | ||
| [OpenTelemetry](https://opentelemetry.io/) is an open-source framework for collecting distributed traces and metrics from applications. It provides a vendor-neutral standard for instrumenting code and exporting telemetry data. | ||
|  | ||
| In Django applications, OpenTelemetry can be used to trace requests through views, middleware, and database operations. The [opentelemetry-instrumentation-django](https://pypi.org/project/opentelemetry-instrumentation-django/) package automatically instruments Django and integrates with the wider OpenTelemetry ecosystem. | ||
|  | ||
| The collected data can be exported to any OpenTelemetry-compatible backend for visualization and analysis. Examples include: | ||
|  | ||
| - Jaeger | ||
| - Grafana Tempo | ||
| - Datadog | ||
| - Elastic APM | ||
|  | ||
| ## Prometheus | ||
|  | ||
| [Prometheus](https://prometheus.io/) is an open-source monitoring system that collects metrics by scraping HTTP endpoints exposed by applications. It stores the data in a time series database and supports flexible querying and alerting. | ||
|  | ||
| For a REST framework project, Prometheus can be used to track metrics such as request counts, error rates, and latency. The [django-prometheus](https://pypi.org/project/django-prometheus/) package adds the necessary instrumentation and exposes a `/metrics` endpoint that Prometheus can scrape. You can also add your own application-specific metrics. | ||
|  | ||
| Prometheus can be paired with [Grafana](https://grafana.com/) to visualize metrics with interactive charts and dashboards. | ||
|  | ||
| ## Other third party packages | ||
|  | ||
| ### Apitally | ||
|  | ||
| [Apitally](https://apitally.io/) is a simple API monitoring, analytics, and request logging tool with an integration for Django REST framework. It provides intuitive dashboards with metrics and insights into API requests, errors, performance and consumers. | ||
|  | ||
| The [apitally](https://pypi.org/project/apitally/) package integrates with Django applications through middleware, which automatically captures metrics for API requests and responses, and synchronizes with Apitally in the background. See DRF-specific setup guide [here](https://docs.apitally.io/frameworks/django-rest-framework). | ||
|  | ||
| ### django-health-check | ||
|  | ||
| [django-health-check](https://pypi.org/project/django-health-check/) provides a set of health check endpoints for Django applications. It allows you to monitor the status of various application components including database connections, cache backends, and custom services. | ||
|  | ||
| The library is helpful for production deployments as it enables load balancers, orchestration systems, and monitoring tools to determine application health. It supports both simple health checks and detailed status reporting for different application components. | ||
|  | ||
| ### Sentry | ||
| There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They're sponsoring us (I think), so that might be a good enough criteria to add them here. While you can self-host, it's also primarily a paid (albeit relatively cheap) service | ||
|  | ||
| [Sentry](https://sentry.io/) is an error monitoring platform that provides real-time error tracking and performance monitoring for applications. It automatically captures exceptions, tracks performance issues, and provides detailed context to help debug problems in production. | ||
|  | ||
| For Django REST framework applications, Sentry can monitor API errors, slow endpoints, and database query performance. The [sentry-sdk](https://pypi.org/project/sentry-sdk/) package provides automatic instrumentation for Django views, middleware, and database operations. | ||
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I appreciate that this is your baby 😃 but I'm a bit reluctant to place some paid services into our docs. If we start to add this one, why not add others as well? Where do we draw the line and stop adding? I don't really fancy maintaining an awsome-observability list as part of the docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I completely understand the concern. That said, I'd argue that Apitally is highly relevant in this context, offers DRF-specific integration and docs, and aligns closely with the needs of the audience likely to visit this page. It's a paid service, but very affordable – similar to Sentry.
I completely understand the need to draw a line in terms of linking paid services though, especially as that's a benefit offered to organizations that sponsor DRF. To help with that, I'll sign Apitally up to a corporate funding plan for DRF. That way, inclusion can be based on clear criteria: