Skip to content

keepalive-improvements.md

Latest commit

 

History

History
207 lines (153 loc) · 7.7 KB

keepalive-improvements.md

File metadata and controls

207 lines (153 loc) · 7.7 KB
Raw

Historical Reference — Superseded by Upstream Merge

The Phase 1-3 keepalive improvements proposed in this document were superseded by the upstream merge from irlserver/main (completed at commit aa66a88). ACK throttling was removed and quality evaluation was enhanced upstream. This document is retained as a historical record of the design exploration. For the current model, see EXTENDED_KEEPALIVE_FIX.md and docs/HOW_IT_WORKS.md.

Keepalive-Based Algorithm Improvements

Overview

This document tracks the implementation of improvements to SRTLA's load balancing and quality evaluation algorithms by leveraging connection information from extended keepalive packets.

Current State

Extended Keepalive Protocol

The codebase already supports extended keepalive packets that include rich connection telemetry from the sender:

typedef struct __attribute__((__packed__)) {
  uint32_t conn_id;
  int32_t window;                    // SRT window size
  int32_t in_flight;                 // Packets currently in flight
  uint32_t rtt_ms;                   // Round-trip time in milliseconds
  uint32_t nak_count;                // NAK (retransmission) count
  uint32_t bitrate_bytes_per_sec;    // Client-side bitrate measurement
} connection_info_t;

Packet Length: 38 bytes (extended keepalive)

Previous Status: This data was only parsed and logged, not used for decision-making.

Current Status: FULLY IMPLEMENTED - All telemetry data is now stored and used for quality assessment.

Location: src/protocol/srtla_handler.cpp (handler), src/quality/quality_evaluator.cpp (evaluation)

Proposed Improvements

Phase 1: RTT-Based Quality Assessment (HIGH PRIORITY)

Rationale: Latency is often a better early indicator of connection problems than bandwidth. High or increasing RTT signals congestion, routing issues, or link instability.

Implementation:

  • Store RTT values in ConnectionStats
  • Track RTT history for trend analysis
  • Add error points based on RTT thresholds
  • Monitor RTT variance (jitter)

Error Point Thresholds:

  • RTT > 500ms: +20 error points
  • RTT > 200ms: +10 error points
  • RTT > 100ms: +5 error points
  • High RTT variance: +10 error points

Status: NOT STARTED

Phase 2: NAK Count Validation (HIGH PRIORITY)

Rationale: The sender's NAK count provides ground truth about packet loss and retransmissions, which is more accurate than receiver-side estimation.

Implementation:

  • Store sender NAK count in ConnectionStats
  • Compare with receiver-side packet loss tracking
  • Use NAK rate (NAKs per packet) for quality scoring
  • Replace or supplement current loss detection

Error Point Thresholds:

  • NAK rate > 20%: +40 error points
  • NAK rate > 10%: +20 error points
  • NAK rate > 5%: +10 error points
  • NAK rate > 1%: +5 error points

Status: NOT STARTED

Phase 3: Window Utilization Analysis (MEDIUM PRIORITY)

Rationale: The ratio of in_flight/window reveals how aggressively the sender is using each connection and can indicate congestion or throttling.

Implementation:

  • Calculate window utilization ratio
  • Detect persistently full windows (congestion)
  • Detect low utilization (client-side issues)
  • Use for advanced load balancing decisions

Analysis:

  • Utilization > 95%: Possible congestion, reduce priority
  • Utilization < 30%: Client throttling, investigate
  • Optimal range: 60-80% utilization

Status: NOT STARTED

Phase 4: Sender Bitrate Validation (LOW PRIORITY)

Rationale: Comparing sender and receiver bitrate measurements can detect path issues and validate metrics.

Implementation:

  • Store sender bitrate in ConnectionStats
  • Compare sender vs receiver measurements
  • Alert on significant discrepancies (>20% difference)
  • Use for debugging and diagnostics

Status: ✅ COMPLETED (2025-12-04)

Implementation Plan (COMPLETED)

Step 1: Data Structure Updates ✅

  • Add keepalive metrics fields to ConnectionStats (receiver_config.h)
    • uint32_t rtt_ms
    • uint32_t rtt_history[RTT_HISTORY_SIZE]
    • uint8_t rtt_history_idx
    • time_t last_keepalive
    • int32_t window
    • int32_t in_flight
    • uint32_t sender_nak_count
    • uint32_t last_sender_nak_count
    • uint32_t sender_bitrate_bps

Step 2: Keepalive Handler Updates ✅

  • Modify SRTLAHandler::handle_keepalive() to store metrics
  • Update connection stats with keepalive data
  • Track timestamp of last keepalive received
  • Add helper functions for RTT history and variance

Step 3: Quality Evaluator Enhancements ✅

  • Add RTT-based error point calculation
  • Add NAK rate error point calculation
  • Add window utilization analysis
  • Add bitrate comparison logic

Step 4: Testing and Validation ⏳

  • Test with simulated high-latency connections
  • Test with varying packet loss scenarios
  • Validate error point calculations
  • Monitor impact on load balancing behavior

Step 5: Documentation ✅

  • Update keepalive-improvements.md with implementation details
  • Document keepalive metrics in technical docs
  • Add configuration parameters
  • Update README.md with new quality metrics

Expected Benefits

  1. Earlier Problem Detection: RTT increases often precede bandwidth degradation
  2. More Accurate Loss Tracking: Sender NAK count is ground truth
  3. Better Load Distribution: Window utilization reveals true connection capacity
  4. Improved Debugging: Bitrate comparison helps diagnose path issues
  5. Reduced Latency: Penalizing high-RTT connections improves stream responsiveness

Configuration Parameters

New parameters to add:

// RTT thresholds (milliseconds), graduated so worse RTT reaches worse tiers
inline constexpr uint32_t RTT_THRESHOLD_MODERATE = 100;  // 100ms  -> +5
inline constexpr uint32_t RTT_THRESHOLD_HIGH = 200;      // 200ms  -> +10
inline constexpr uint32_t RTT_THRESHOLD_CRITICAL = 500;  // 500ms  -> +20
inline constexpr uint32_t RTT_THRESHOLD_SEVERE = 1000;   // 1000ms -> +30
inline constexpr uint32_t RTT_THRESHOLD_EXTREME = 2000;  // 2000ms -> +40

// Window utilization thresholds
inline constexpr double WINDOW_UTILIZATION_CONGESTED = 0.95;
inline constexpr double WINDOW_UTILIZATION_LOW = 0.30;

// Bitrate comparison tolerance
inline constexpr double BITRATE_DISCREPANCY_THRESHOLD = 0.20;  // 20%

// Jitter detection — stddev scored relative to mean RTT (coefficient of
// variation), not an absolute stddev, so normal cellular jitter on a healthy
// link is not penalized. Replaces the retired absolute RTT_VARIANCE_THRESHOLD.
inline constexpr double RTT_JITTER_RATIO_HIGH = 1.0;     // stddev > 1.0*mean -> +5
inline constexpr double RTT_JITTER_RATIO_SEVERE = 1.5;   // stddev > 1.5*mean -> +10

Risks and Mitigations

Risk: Keepalive packets might not arrive regularly

  • Mitigation: Only apply RTT-based penalties if keepalive received within last 2 seconds
  • Mitigation: Fall back to receiver-side metrics if keepalives stale

Risk: Sender-side metrics might be inaccurate

  • Mitigation: Use as supplementary data, not sole decision factor
  • Mitigation: Validate against receiver measurements

Risk: Too aggressive RTT penalties might exclude viable connections

  • Mitigation: Use gradual error point increases, not binary decisions
  • Mitigation: Maintain grace period for new connections

Progress Tracking

  • Phase 1 (RTT): ✅ 100% complete
  • Phase 2 (NAK): ✅ 100% complete
  • Phase 3 (Window): ✅ 100% complete
  • Phase 4 (Bitrate): ✅ 100% complete

Overall Progress: ✅ 100% (Implementation Complete)

Implementation Date: 2025-12-04 Build Status: ✅ Successful Next Steps: Testing and validation