Project Hydra

A real-time stock monitoring platform that heals itself.

Hydra pairs a live market-data dashboard with a Kubernetes cluster whose recovery decisions are made entirely by a trained reinforcement learning agent — no LivenessProbe, ReadinessProbe, or HorizontalPodAutoscaler involved. Every restart, scale-up, and scale-down is a PPO policy decision, stress-tested against real failure injected by Chaos Mesh. The goal: a system that cannot afford downtime, recovering from CPU exhaustion, memory leaks, and upstream latency spikes without a human — or a static rule — in the loop.

This project is a hands-on implementation of the self-healing architecture described in AI Meets Chaos Engineering by Dhruv Mistry: instrument the system, define a bounded set of healing actions, close the loop with an RL agent, and use chaos injection as both a test harness and a training signal.

---

How it works

┌─────────────┐     ┌──────────────┐     ┌───────────────────┐
│   Frontend   │────▶│  Hydra API   │────▶│   Yahoo Finance    │
│   (React)    │◀────│  (FastAPI)   │     └───────────────────┘
└─────────────┘     └──────┬───────┘
                            │ /metrics
                            ▼
                    ┌──────────────┐
                    │  Prometheus   │◀──────────┐
                    └──────┬───────┘            │
                            │ state              │ chaos
                            ▼                    │
                    ┌──────────────┐     ┌───────┴────────┐
                    │  RL Controller│────▶│  Chaos Mesh    │
                    │  (observe/    │     │  (CPU/mem/net) │
                    │   decide/act) │     └────────────────┘
                    └──────┬───────┘
                            │ predict
                            ▼
                    ┌──────────────┐
                    │  PPO Agent    │
                    │  (FastAPI)    │
                    └──────────────┘

1. Hydra backend (hydra/) — a FastAPI service that pulls OHLCV candles from Yahoo Finance via yfinance, computes SMA-20, SMA-50, and EWM-annualized volatility, and serves it as JSON. It also exports its own cgroup-level CPU and memory usage as Prometheus metrics, so the RL agent is watching the container's actual resource pressure, not a proxy for it.
2. Frontend (frontend/) — a React + Vite dashboard that polls the backend every 20 seconds and renders synchronized candlestick, moving-average, volume, and volatility panels using lightweight-charts.
3. RL Controller (rl_controller/) — the brain of the system. Every 15 seconds it pulls a 9-dimensional state vector from Prometheus (pod health, CPU, memory, latency, error rate, time-since-failure, replica count), sends it to the PPO model for a prediction, applies a small set of hard safety guardrails, and executes the resulting action directly against the Kubernetes API — restart a pod, scale up, scale down, or do nothing.
4. Chaos Mesh (chaosmesh/) — injects CPU stress, memory stress, network delay, and HTTP latency against the Hydra deployment on a fixed interval, giving the agent (and you) a continuous stream of real failure to respond to.
5. Prometheus + Grafana — scrape and visualize everything: container metrics, HTTP latency histograms, and the RL agent's own decisions (hydra_rl_last_action, hydra_rl_chaos_active) as first-class time series.

The training environment itself (Hydra_RL_sim_env_Final.ipynb) is a custom gymnasium environment that simulates pods under variable request load, with a reward function that penalizes crashes, over-provisioning, under-provisioning, and action oscillation — the resulting policy (ppo_kubernetes_hardened.zip) is what the live controller loads and serves.

Features

• Zero native Kubernetes self-healing — LivenessProbe, ReadinessProbe, and HPA are deliberately absent. Recovery is 100% agent-driven, evaluated every 30 seconds.
• Live stock dashboard — candlesticks, SMA-20/50, volume, and EWM volatility for any ticker, with configurable interval/period combinations and automatic fallback messaging when Yahoo Finance throttles or returns a derived interval.
• Custom Prometheus exporters — cgroup v2 (with v1 fallback) CPU and memory collectors built from scratch by reading /sys/fs/cgroup directly, rather than relying on cAdvisor or kube-state-metrics alone.
• PPO-based decision loop — a 4-action policy space (no-op, restart pod, scale up, scale down) trained in a simulated Kubernetes environment, served over a /predict FastAPI endpoint, and wired into a live observe → decide → act loop.
• Hard safety guardrails around the learned policy — the controller doesn't trust the model blindly. It forces a restart if memory is clearly leaking while CPU sits idle, and forces the system back to a sane replica count if the agent proposes an unnecessary scale action while the system is idle and correctly sized.
• Automated, cycling chaos injection — the controller itself toggles Chaos Mesh experiments on a timer, alternating between an injected-failure phase and an idle recovery phase, so resilience is continuously exercised rather than tested once.
• Full observability stack — a provisioned Grafana dashboard, Prometheus scrape config driven by Kubernetes service discovery and RBAC, and structured request logging with request-ID correlation on every HTTP call.
• Ingress-level resilience — the nginx Ingress is configured to retry against the next healthy pod on connection errors, timeouts, and 5xx responses, with bounded retry counts and wall-clock timeouts, so a mid-recovery pod doesn't surface as a failed request to the user.

Installation

Prerequisites

• Minikube (or another Kubernetes cluster) with the Docker driver
• kubectl and helm
• kustomize (bundled with recent kubectl via kubectl apply -k)
• Docker

Setup

# 1. Start the cluster
minikube start

# 2. Build the service images (Hydra backend, frontend, RL controller)
#    into minikube's Docker daemon
chmod +x scripts/build.sh
./scripts/build.sh

# 3. Install ingress-nginx and Chaos Mesh into the cluster
chmod +x scripts/setup.sh
./scripts/setup.sh

# 4. Apply all manifests via kustomize and start the tunnel
chmod +x scripts/start.sh
./scripts/start.sh

start.sh runs minikube tunnel in the foreground — leave it running in its own terminal. In a second terminal:

# 5. Get the external IP of the ingress controller
kubectl get svc -n ingress-nginx

# 6. Point rtsm.com at that IP
echo "<EXTERNAL-IP>  rtsm.com" | sudo tee -a /etc/hosts

Once the tunnel is up and /etc/hosts is updated, http://rtsm.com serves the dashboard.

Usage

Route	Serves
http://rtsm.com/	Stock monitoring dashboard (React frontend)
http://rtsm.com/data?ticker=AAPL&period=1d&interval=1m	Raw OHLCV + indicator JSON from the Hydra backend
http://rtsm.com/metrics	Hydra's Prometheus metrics (cgroup CPU/mem, HTTP latency)
http://rtsm.com/health	Liveness of the Yahoo Finance upstream
http://rtsm.com/dashboard	Grafana, provisioned with the Prometheus datasource and dashboard JSON in this repo

To watch the self-healing loop in action, open Grafana and the RL controller logs side by side:

kubectl logs -n hydra -f deployment/rl-controller

You'll see the observe → decide → act cycle every 15 seconds, chaos injection/removal every 120 seconds, and any guardrail overrides logged explicitly (e.g. GUARDRAIL: memory leak detected. Forcing [ 1 : Restart-Pod ]).

Manually injecting chaos

The chaosmesh/ YAMLs can also be applied by hand, outside the controller's automatic cycle, to test a specific scenario in isolation:

kubectl apply -f chaosmesh/cpu-stress.yaml     # 80% CPU load on one pod for 5m
kubectl apply -f chaosmesh/mem-stress.yaml     # 200MiB memory stress for 5m
kubectl apply -f chaosmesh/network-stress.yaml # 2s network delay for 10m
kubectl apply -f chaosmesh/http-delay.yaml     # 2s delay on Yahoo Finance calls for 10m

Retraining the policy

Hydra_RL_sim_env_Final.ipynb contains the full gymnasium environment and PPO training loop (via stable-baselines3). Running it end-to-end regenerates a .zip policy compatible with rl_agent_service.py — swap it in via the MODEL_PATH environment variable or by replacing ppo_kubernetes_hardened.zip in rl_controller/.

Configuration

Most runtime tuning lives in rl_controller/config.yaml:

observation_interval: 15        # seconds between observe/decide/act cycles
chaos_injection_interval: 120   # seconds between toggling chaos on/off
max_replicas: 10
min_replicas: 1
rl_agent_service_url: "http://rl-agent-service:8000"
prometheus_url: "http://prometheus-service:9090"
namespace: "hydra"
deployment_name: "hydra-deployment"
max_steps: 200

Other configuration surfaces:

• RL agent model — rl_agent_service.py reads MODEL_PATH (defaults to ppo_kubernetes_hardened.zip); if the model fails to load, the service falls back to always returning a no-op action rather than crashing.
• Chaos experiment set — the controller cycles through whatever .yaml files exist in CHAOS_DIR (defaults to ../chaosmesh), so adding or removing experiment files changes what gets injected without a code change.
• Resource limits — kb8-configs/hydra-depl.yaml caps the Hydra backend at 512Mi memory / 500m CPU (requesting 256Mi / 100m), which is what the RL agent is implicitly learning to stay under.
• Ingress retry behavior — timeout and retry-count tuning lives in the annotations on kb8-configs/ingress.yaml.

Testing

There isn't a conventional unit test suite — validation here is closed-loop and empirical:

1. Simulated training validation — the notebook trains against a custom gymnasium environment with injected pod crashes, memory leaks, and load spikes, and logs learning curves (simple_learning_curve.png, training_convergence.png) to confirm the policy actually converges before it's ever pointed at a real cluster.
2. Live chaos validation — with the cluster running, Chaos Mesh injects real CPU/memory/network/HTTP failures on a fixed cadence, and correctness is judged by whether the controller's chosen action (visible in its logs and in Grafana via hydra_rl_last_action) matches the expected recovery response for that failure mode.
3. Guardrail testing — because the guardrails in chaos_rl_controller.py are simple, explicit conditionals (e.g. force a restart when memory is high but CPU is idle), they can be exercised directly by injecting the corresponding Chaos Mesh experiment and confirming the override fires in the logs rather than the raw model prediction.

See evaluation_report.md for a full write-up of agent behavior observed during chaos testing, including recovery actions per failure type and a comparison against native HPA-based scaling.

Acknowledgements

• Built for IEEE NITK CompSoc.
• Architecture inspired by AI Meets Chaos Engineering: Designing Self-Healing Systems using Reinforcement Learning by Dhruv Mistry.
• RL training powered by Stable-Baselines3 and Gymnasium.
• Chaos injection via Chaos Mesh.
• Market data via yfinance (Yahoo Finance).
• Charting via lightweight-charts (TradingView).