WebSocket Connection Health Scoring with Tiered Reconnection Strategy

[JamesEjembi]

Problem Statement / Feature Objective

The dashboard depends on persistent WebSocket connections to beacon nodes for real-time validator updates. Network partitions, node restarts, and rate-limiting cause intermittent disconnections that current exponential-backoff reconnection cannot distinguish (transient vs. permanent failure). Build a health scoring system that assigns a real-time 0–100 health score to each WS connection, classifies disconnection reasons, and applies a tiered reconnection strategy: Tier 1 (immediate, <5s) for transient blips, Tier 2 (exponential, 5–300s) for network instability, Tier 3 (manual intervention required) for auth/version mismatches.

Technical Invariants & Bounds

• Health score = 50 × (uptime_ratio_last_60s) + 30 × (avg_message_latency_ms / 1000, clamped 0–1 inverted) + 20 × (consecutive_reconnects, decaying: max at 0 reconnects)
• Disconnection classification using WebSocket close codes: 1000 (normal, Tier 1), 1006/1015 (abnormal, Tier 2), 4000–4009 (auth error, Tier 3), 3000–3009 (version mismatch, Tier 3)
• Tier 1: reconnect immediately, max 3 attempts before escalating to Tier 2
• Tier 2: exponential backoff 5s → 15s → 45s → 135s (×3 multiplier), max 300s, resets after 5min stable connection
• Tier 3: alert operator via notification bar, disable auto-reconnect, provide "Retry" button
• Health history: rolling 1-hour buffer of health snapshots at 5s intervals stored in memory (720 entries per connection)
• Multi-connection support: up to 10 simultaneous WS connections with independent health scoring

Codebase Navigation Guide

• src/services/webSocketManager.ts – existing WS manager, refactor with health scoring
• src/hooks/useWebSocketHealth.ts – new hook for health data consumption
• src/components/network/WSHealthDashboard.tsx – connection health visualization
• src/utils/connectionClassifier.ts – close code → tier mapping
• src/utils/exponentialBackoff.ts – backoff calculator with jitter
• src/store/networkSlice.ts – network state management

Implementation Blueprint

1. Create src/utils/connectionClassifier.ts – maps WS close codes to Tiers 1–3 with regex matching for custom code ranges
2. Build src/utils/exponentialBackoff.ts – computes next delay using ×3 multiplier + 500ms random jitter, caps at 300s
3. Implement src/utils/healthScore.ts – computes health score from uptime ratio, latency, and reconnect count using the weighted formula
4. Refactor src/services/webSocketManager.ts – integrate health scoring into existing WS lifecycle:
   • Track uptime per connection (performance.now() deltas)
   • Track message latency (timestamp sent vs received for heartbeat pings)
   • On close: classify code → apply tiered reconnection
   • On open: reset reconnect counter if stable for 5min
5. Create src/hooks/useWebSocketHealth.ts – exposes per-connection health scores, tier status, and reconnect attempts
6. Build src/components/network/WSHealthDashboard.tsx – renders connection table with health bar (green/yellow/red), tier badge, latency sparkline, reconnect count
7. Mount in navigation sidebar or src/components/layout/StatusBar.tsx