Skip to content

Commit f892ea9

Browse files
authored
refactor: complete decentralization - remove all centralized infrastructure (#6)
* refactor: complete decentralization - remove all centralized infrastructure BREAKING CHANGE: Platform is now fully P2P Removed centralized components: - crates/platform-server/ (central API server with PostgreSQL) - bins/platform/ (unified binary with server/validator modes) - bins/csudo/ (privileged command execution) - bins/validator-decentralized/ (renamed to validator-node) - Dockerfile.server, Dockerfile.full, Dockerfile.broker - docker-compose.server.yml - .env.server.example - crates/challenge-sdk/src/platform_client.rs (WebSocket client) Infrastructure changes: - Renamed Dockerfile.decentralized to Dockerfile - Renamed docker-compose.decentralized.yml to docker-compose.yml - Updated Cargo.toml workspace members - Fixed Dockerfile permissions (755 instead of 777) - Added security documentation for privileged mode Code updates: - Removed centralized exports from challenge-sdk - Updated docker.rs comments for decentralized architecture - Fixed 'centralized' wording in challenge_store.rs Architecture is now fully decentralized: - Validators communicate via libp2p gossipsub - State stored in distributed DHT - No central server dependency * docs: update documentation for P2P decentralized architecture - Update AGENTS.md to reflect fully decentralized P2P network - Update README.md for validator-only deployment - Remove references to centralized infrastructure - Add P2P architecture diagrams and explanations
1 parent 63a6e38 commit f892ea9

54 files changed

Lines changed: 639 additions & 14428 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.server.example

Lines changed: 0 additions & 56 deletions
This file was deleted.

.github/workflows/release.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,6 @@ jobs:
4242
run: |
4343
mkdir -p release
4444
cp target/release/validator-node release/ 2>/dev/null || true
45-
cp target/release/platform-server release/ 2>/dev/null || true
4645
cp target/release/csudo release/ 2>/dev/null || true
4746
tar -czvf platform-${{ needs.release-please.outputs.version }}-linux-x86_64.tar.gz -C release .
4847

AGENTS.md

Lines changed: 58 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ This document explains how agents (miners) interact with the Platform network.
66

77
## Important: Challenge-Specific Repositories
88

9-
**Platform is a load balancer and orchestration layer.** It does not contain challenge-specific logic.
9+
**Platform is a fully decentralized P2P network for distributed evaluation.** It does not contain challenge-specific logic.
1010

1111
Each challenge has its own repository with:
1212
- Task definitions and evaluation criteria
@@ -25,26 +25,34 @@ Each challenge has its own repository with:
2525

2626
## What is Platform?
2727

28-
Platform is infrastructure that:
28+
Platform is a **fully decentralized P2P infrastructure** that:
2929

30-
1. **Routes submissions** from miners to validators
31-
2. **Orchestrates evaluation** across distributed validators
32-
3. **Aggregates scores** using stake-weighted consensus
30+
1. **Propagates submissions** from miners across the validator network via gossipsub
31+
2. **Orchestrates evaluation** across distributed validators using DHT coordination
32+
3. **Aggregates scores** using stake-weighted consensus (P2P)
3333
4. **Submits weights** to Bittensor at epoch boundaries
3434

3535
```
36-
┌─────────────┐ ┌──────────────────┐ ┌─────────────┐
37-
│ MINERS │ ──▶ │ PLATFORM │ ──▶ │ BITTENSOR │
38-
│ (Agents) │ │ (Load Balancer) │ │ (Weights) │
39-
└─────────────┘ └──────────────────┘ └─────────────┘
40-
41-
┌──────┴──────┐
42-
▼ ▼
43-
┌──────────┐ ┌──────────┐
44-
│Challenge │ │Challenge │
45-
│ A │ │ B │
46-
│ (Repo) │ │ (Repo) │
47-
└──────────┘ └──────────┘
36+
┌─────────────┐ ┌─────────────┐
37+
│ MINERS │ ──────────────────────────▶ │ BITTENSOR │
38+
│ (Agents) │ │ (Weights) │
39+
└─────────────┘ └─────────────┘
40+
│ ▲
41+
│ P2P (libp2p) │
42+
▼ │
43+
┌──────────────────────────────────────────────────────────┐
44+
│ VALIDATOR P2P NETWORK │
45+
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
46+
│ │ Validator │◀─▶│ Validator │◀─▶│ Validator │ ... │
47+
│ │ Node │ │ Node │ │ Node │ │
48+
│ └───────────┘ └───────────┘ └───────────┘ │
49+
│ │ │ │ │
50+
│ ▼ ▼ ▼ │
51+
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
52+
│ │Challenge │ │Challenge │ │Challenge │ │
53+
│ │Container │ │Container │ │Container │ │
54+
│ └──────────┘ └──────────┘ └──────────┘ │
55+
└──────────────────────────────────────────────────────────┘
4856
```
4957

5058
---
@@ -69,10 +77,10 @@ class MyAgent(Agent):
6977

7078
### 2. Submission
7179

72-
Submit your agent code to the Platform API. The submission is:
73-
- Stored in the central database
80+
Submit your agent code to the P2P network. The submission is:
81+
- Broadcast to validators via libp2p gossipsub
7482
- Compiled/validated by the challenge system
75-
- Queued for evaluation by validators
83+
- Distributed across the validator network for evaluation
7684

7785
### 3. Evaluation
7886

@@ -83,10 +91,10 @@ Validators independently evaluate your submission:
8391

8492
### 4. Scoring
8593

86-
Platform aggregates validator scores:
87-
- Stake-weighted averaging
94+
Validators aggregate scores across the P2P network:
95+
- Stake-weighted averaging via DHT coordination
8896
- Outlier detection (removes anomalous validators)
89-
- Confidence calculation
97+
- Consensus achieved through gossipsub protocol
9098

9199
### 5. Rewards
92100

@@ -96,23 +104,32 @@ At epoch end (~72 minutes), weights are submitted to Bittensor:
96104

97105
---
98106

99-
## Platform API
107+
## P2P Network
100108

101-
### Endpoints
109+
### How It Works
102110

103-
All challenge-specific endpoints are proxied through Platform:
111+
Platform uses libp2p for fully decentralized communication:
104112

105-
| Endpoint | Description |
106-
|----------|-------------|
107-
| `POST /api/v1/bridge/{challenge_id}/submit` | Submit agent code |
108-
| `GET /api/v1/bridge/{challenge_id}/status/{hash}` | Check submission status |
109-
| `GET /api/v1/bridge/{challenge_id}/leaderboard` | View rankings |
113+
- **Gossipsub**: Submissions and scores are broadcast across the validator network
114+
- **DHT (Kademlia)**: Peer discovery and coordination without central servers
115+
- **Direct Connections**: Validators communicate directly with each other
110116

111117
### Authentication
112118

113-
Submissions are signed with your Bittensor hotkey:
119+
All P2P messages are signed with your Bittensor hotkey:
114120
- Use `sr25519` signature scheme
115121
- Include timestamp to prevent replay attacks
122+
- Validators verify signatures before processing
123+
124+
### Submitting via P2P
125+
126+
Connect to any validator node to submit your agent:
127+
```bash
128+
# Using the challenge CLI (recommended)
129+
term submit --agent my_agent.py --peer /ip4/VALIDATOR_IP/tcp/9000/p2p/PEER_ID
130+
131+
# Or via the challenge SDK
132+
```
116133

117134
---
118135

@@ -131,7 +148,7 @@ Check the challenge repository for:
131148

132149
### How are scores calculated?
133150

134-
Each challenge defines its own scoring algorithm. Platform only aggregates scores from validators.
151+
Each challenge defines its own scoring algorithm. Validators coordinate score aggregation via P2P consensus.
135152

136153
### Can I test locally?
137154

@@ -152,14 +169,12 @@ Defined by each challenge. Check the challenge docs for specific limits.
152169
```
153170
Platform Repository (this repo)
154171
├── crates/
155-
│ ├── platform-server/ # Central coordination server
156-
│ ├── challenge-sdk/ # SDK for building challenges (not agents!)
157-
│ ├── challenge-orchestrator/
158-
│ ├── bittensor-integration/
172+
│ ├── challenge-sdk/ # SDK for building challenges (not agents!)
173+
│ ├── challenge-orchestrator/ # Manages challenge container execution
174+
│ ├── bittensor-integration/ # Bittensor network integration
159175
│ └── ...
160176
└── bins/
161-
├── platform/ # Server binary
162-
└── validator-node/ # Validator binary
177+
└── validator-node/ # P2P validator node binary
163178
164179
Challenge Repositories (separate)
165180
├── term-challenge/ # Terminal Bench
@@ -169,6 +184,8 @@ Challenge Repositories (separate)
169184
└── (other challenges)/
170185
```
171186

187+
**Note:** Platform is fully decentralized—there is no central server. All validators communicate directly via libp2p (gossipsub + DHT).
188+
172189
---
173190

174191
## Getting Started
@@ -177,7 +194,7 @@ Challenge Repositories (separate)
177194
2. **Go to that challenge's repository** (not this one)
178195
3. **Read the challenge-specific documentation**
179196
4. **Develop your agent** using the challenge SDK
180-
5. **Submit** through the Platform API or challenge CLI
197+
5. **Submit** through the P2P network or challenge CLI
181198
6. **Monitor** your submission status and leaderboard position
182199

183200
---
@@ -187,7 +204,7 @@ Challenge Repositories (separate)
187204
- [Bittensor Docs](https://docs.bittensor.com) - Network documentation
188205
- [Validator Guide](docs/validator.md) - Running a validator
189206

190-
**Note:** In P2P mode (recommended), validators communicate directly without a central server.
207+
Platform is fully decentralized—validators communicate directly via P2P without any central server.
191208
See the main README for deployment instructions.
192209

193210
For challenge-specific questions, please refer to the appropriate challenge repository.

0 commit comments

Comments
 (0)