Skip to content

Commit 8c38351

Browse files
vex0209-btvex0209-bt
committed
init commit
0 parents  commit 8c38351

98 files changed

Lines changed: 19952 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Git and env
2+
.git
3+
.gitignore
4+
.env
5+
.env.*
6+
7+
# Python
8+
__pycache__
9+
*.py[cod]
10+
*.pyo
11+
.venv
12+
venv
13+
*.egg-info
14+
.eggs
15+
16+
# Tests and tooling (not needed in image)
17+
tests
18+
.pytest_cache
19+
.coverage
20+
htmlcov
21+
.mypy_cache
22+
23+
# Docs and compose (optional; keep if you want them in image)
24+
# leoma-docs
25+
docker-compose*.yml
26+
Dockerfile*
27+
.dockerignore
28+
29+
# IDE and OS
30+
.idea
31+
.vscode
32+
.DS_Store
33+
Thumbs.db

‎.github/workflows/docker-publish.yml‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# Leoma Subnet - Docker Build and Push
2+
#
3+
# This workflow builds and pushes the Docker image to Docker Hub
4+
# whenever code is pushed to the main branch.
5+
#
6+
# Validators using Watchtower will automatically pull the new image.
7+
#
8+
# Setup:
9+
# 1. Create a Docker Hub account and repository
10+
# 2. Create an access token at https://hub.docker.com/settings/security
11+
# 3. Add secrets to your GitHub repo (Settings > Secrets > Actions):
12+
# - DOCKERHUB_USERNAME: Your Docker Hub username
13+
# - DOCKERHUB_TOKEN: Your Docker Hub access token
14+
15+
name: Build and Push Docker Image
16+
17+
on:
18+
push:
19+
branches:
20+
- main
21+
- master
22+
# Only rebuild when relevant files change
23+
paths:
24+
- 'leoma.py'
25+
- 'pyproject.toml'
26+
- 'Dockerfile'
27+
- '.github/workflows/docker-publish.yml'
28+
29+
# Allow manual trigger for testing
30+
workflow_dispatch:
31+
32+
env:
33+
REGISTRY: docker.io
34+
# Change this to your Docker Hub username and repo name
35+
# Format: username/repository
36+
IMAGE_NAME: ${{ github.repository }}
37+
38+
jobs:
39+
build-and-push:
40+
runs-on: ubuntu-latest
41+
42+
permissions:
43+
contents: read
44+
packages: write
45+
46+
steps:
47+
- name: Checkout repository
48+
uses: actions/checkout@v4
49+
50+
- name: Set up QEMU
51+
uses: docker/setup-qemu-action@v3
52+
53+
- name: Set up Docker Buildx
54+
uses: docker/setup-buildx-action@v3
55+
56+
- name: Log in to Docker Hub
57+
uses: docker/login-action@v3
58+
with:
59+
username: ${{ secrets.DOCKERHUB_USERNAME }}
60+
password: ${{ secrets.DOCKERHUB_TOKEN }}
61+
62+
- name: Extract metadata for Docker
63+
id: meta
64+
uses: docker/metadata-action@v5
65+
with:
66+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
67+
tags: |
68+
# Tag as 'latest' for default branch
69+
type=raw,value=latest,enable={{is_default_branch}}
70+
# Tag with git SHA for rollback capability
71+
type=sha,prefix=
72+
# Tag with branch name
73+
type=ref,event=branch
74+
# Tag with version if tagged
75+
type=semver,pattern={{version}}
76+
type=semver,pattern={{major}}.{{minor}}
77+
78+
- name: Build and push Docker image
79+
uses: docker/build-push-action@v5
80+
with:
81+
context: .
82+
push: true
83+
tags: ${{ steps.meta.outputs.tags }}
84+
labels: ${{ steps.meta.outputs.labels }}
85+
# Cache layers for faster builds
86+
cache-from: type=gha
87+
cache-to: type=gha,mode=max
88+
# Build for both AMD64 and ARM64
89+
platforms: linux/amd64,linux/arm64
90+
91+
- name: Generate build summary
92+
run: |
93+
echo "## Docker Image Published 🐳" >> $GITHUB_STEP_SUMMARY
94+
echo "" >> $GITHUB_STEP_SUMMARY
95+
echo "**Image:** \`${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}\`" >> $GITHUB_STEP_SUMMARY
96+
echo "" >> $GITHUB_STEP_SUMMARY
97+
echo "**Tags:**" >> $GITHUB_STEP_SUMMARY
98+
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
99+
echo "${{ steps.meta.outputs.tags }}" >> $GITHUB_STEP_SUMMARY
100+
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
101+
echo "" >> $GITHUB_STEP_SUMMARY
102+
echo "Validators running with Watchtower will automatically update within 5 minutes." >> $GITHUB_STEP_SUMMARY

‎.gitignore‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Python-generated files
2+
__pycache__/
3+
*.py[oc]
4+
build/
5+
dist/
6+
wheels/
7+
*.egg-info
8+
9+
# Virtual environments
10+
.venv
11+
venv/
12+
13+
# Environment files (contain secrets)
14+
.env
15+
.env.local
16+
.env.*.local
17+
18+
# Docker volumes
19+
validator-data/
20+
21+
# Bittensor wallets (never commit these!)
22+
wallets/
23+
.bittensor/
24+
25+
# IDE
26+
.idea/
27+
.vscode/
28+
*.swp
29+
*.swo
30+
31+
# OS
32+
.DS_Store
33+
Thumbs.db
34+
35+
# Node
36+
node_modules/
37+
38+
.pytest_cache/

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.12

‎Dockerfile‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
FROM python:3.12-slim
2+
3+
# ffprobe/ffmpeg for video processing (evaluator, owner-sampler); curl for healthchecks; build-essential for compiling Python deps if needed.
4+
RUN apt-get update && apt-get install -y --no-install-recommends \
5+
ffmpeg curl build-essential \
6+
&& rm -rf /var/lib/apt/lists/*
7+
8+
WORKDIR /app
9+
RUN pip install --no-cache-dir uv
10+
11+
COPY pyproject.toml README.md ./
12+
COPY leoma.py ./
13+
COPY leoma ./leoma
14+
15+
# Install package (non-editable for production image)
16+
RUN uv pip install --system --no-cache .
17+
18+
# Override in compose: leoma serve (validator) or leoma api (API service)
19+
CMD ["leoma"]

‎README.md‎

Lines changed: 189 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,189 @@
1+
# Leoma Subnet
2+
3+
Leoma is an **AI video subnet** on [Bittensor](https://docs.learnbittensor.org/). Miners run **Text-Image to Video (TI2V)** models; validators evaluate miner outputs and set **winner-take-all** weights on-chain. The best miner earns subnet alpha each round.
4+
5+
**Supported model type (current):** **Text-Image to Video (TI2V)** only.
6+
7+
**Roadmap:** Support for **Text-to-Video (T2V)** and **Image-to-Video (I2V)** is planned.
8+
9+
---
10+
11+
## Contents
12+
13+
- [What is Leoma?](#what-is-leoma)
14+
- [Workflow](#workflow)
15+
- [Validator setup](#validator-setup)
16+
- [Miner setup](#miner-setup)
17+
- [Storage and API](#storage-and-api)
18+
- [Security and production](#security-and-production)
19+
- [Documentation](#documentation)
20+
- [License](#license)
21+
22+
---
23+
24+
## What is Leoma?
25+
26+
Leoma is a **Bittensor subnet** for **AI-generated video**:
27+
28+
- **Current support:** **Text-Image to Video (TI2V)** — validators send a first frame (image) and a text prompt; miners return a short video. Validators score generated videos with a strict multi-aspect benchmark prompt (first-frame fidelity, prompt adherence, temporal quality, visual artifacts) and record pass/fail wins. Ranking is winner-take-all; the top miner gets full weight each round.
29+
- **Roadmap:** **Text-to-Video (T2V)** and **Image-to-Video (I2V)** support are planned.
30+
31+
## Workflow
32+
33+
1. **Subnet owner** runs the **owner-sampler**: creates tasks (first frame + prompt from one-shot 5s clips in Hippius S3), calls miners via Chutes, uploads task artifacts to the samples bucket, and sets the latest task id on the API.
34+
2. **Validators** run the **evaluator** and **weight-setter**: poll the API for the latest task, download task data from S3, run GPT-4o evaluation, POST results to the Leoma API; each epoch, call **GET /weights** and set on-chain weights (winner-take-all).
35+
3. The **API** (subnet owner) computes rank (dominance rule) and exposes **GET /weights**. Validators use this to set weights on-chain.
36+
4. **Miners** register a Hugging Face model (naming: `leoma` prefix, hotkey suffix) and Chute endpoint via on-chain commit. They receive challenges; the best performer earns subnet alpha.
37+
38+
| Role | In Leoma |
39+
|------|----------|
40+
| **Miner** | Upload a TI2V model to Hugging Face (name: `leoma...` + your hotkey), deploy to Chutes, commit on-chain. Earn subnet alpha when your outputs win. |
41+
| **Validator** | Run evaluator + weight-setter (e.g. `leoma serve`). Requires API URL, Hippius S3 read (samples bucket), OpenAI API key, and Bittensor wallet. |
42+
43+
---
44+
45+
## Validator setup
46+
47+
Validators run the **evaluator** and the **weight-setter**. Task creation and miner calls are done by the subnet owner (owner-sampler), not validators.
48+
49+
### Prerequisites
50+
51+
- **Bittensor wallet** (coldkey + hotkey) registered as a validator on the Leoma subnet.
52+
- **Leoma API** URL (the deployed owner API).
53+
- **Hippius S3** access: **read-only** to the **samples** bucket (evaluator downloads task data from S3; evaluation results are submitted to the Leoma API with hotkey signature). Use [access keys](https://docs.hippius.com/storage/s3/integration) from the Hippius console.
54+
- **OpenAI API key** (for GPT-4o evaluation in the evaluator).
55+
- **Validator registration in API DB:** An admin must add your validator hotkey (and UID, stake) so the API includes you in stake-weighted scoring (e.g. `leoma db add-validator --uid <uid> --hotkey <ss58>`).
56+
57+
### Environment variables
58+
59+
Set these in `.env` (copy from `env.example`):
60+
61+
| Variable | Description |
62+
|----------|-------------|
63+
| `API_URL` | Leoma API base URL (e.g. `https://api.leoma.ai`) |
64+
| `NETUID` | Subnet ID (e.g. `99`) |
65+
| `NETWORK` | Bittensor network (`finney` for mainnet) |
66+
| `WALLET_NAME` | Bittensor wallet name (e.g. `default`) |
67+
| `HOTKEY_NAME` | Bittensor hotkey name (e.g. `default`) |
68+
| `OPENAI_API_KEY` | OpenAI API key for GPT-4o (evaluator) |
69+
| `EPOCH_LEN` | Blocks per epoch (e.g. `180`); optional |
70+
| `HIPPIUS_ENDPOINT` | e.g. `s3.hippius.com` |
71+
| `HIPPIUS_REGION` | e.g. `decentralized` |
72+
| `HIPPIUS_SAMPLES_BUCKET` | Samples bucket (e.g. `samples`) |
73+
| `HIPPIUS_SAMPLES_READ_ACCESS_KEY` / `HIPPIUS_SAMPLES_READ_SECRET_KEY` | Read access to samples bucket (evaluator downloads tasks only; no write needed) |
74+
75+
### Quick start (Docker, recommended)
76+
77+
This repo’s `docker-compose.yml` runs the **validator** (evaluator + weight-setter in one container) and optionally **Watchtower** for auto-updates.
78+
79+
```bash
80+
# 1. Clone the repo
81+
git clone https://github.com/RendixNetwork/leoma.git
82+
cd leoma
83+
84+
# 2. Create .env from example
85+
cp env.example .env
86+
# Edit .env: API_URL, OPENAI_API_KEY, HIPPIUS_SAMPLES_READ_*, WALLET_NAME, HOTKEY_NAME, NETUID, NETWORK
87+
88+
# 3. Run
89+
docker compose up -d
90+
```
91+
92+
This starts **leoma-validator** (`leoma serve`: evaluator + weight-setter) and **leoma-watchtower**. Mount your Bittensor wallets so the container can sign weight-setting transactions; the compose file uses `~/.bittensor/wallets:/root/.bittensor/wallets:ro`.
93+
94+
**Auto-update with Watchtower:** Build and push the image on subnet code updates; Watchtower will pull and restart the validator container. See `env.example` for `WATCHTOWER_POLL_INTERVAL` and related options.
95+
96+
### Manual installation
97+
98+
```bash
99+
# Requires Python 3.12+
100+
pip install -e . # or: uv pip install -e .
101+
102+
# Run validator (evaluator + weight-setter in one process)
103+
leoma serve
104+
```
105+
106+
### Split processes (advanced)
107+
108+
You can run evaluator and weight-setter as separate processes:
109+
110+
```bash
111+
leoma servers evaluator # Polls GET /tasks/latest, downloads from S3, GPT-4o, POSTs to API
112+
leoma servers validator # Every epoch: GET /weights, set on-chain
113+
```
114+
115+
### API authentication
116+
117+
Endpoints that require validator identity (e.g. `POST /samples/batch`) use **signature auth**. Send headers: `X-Validator-Hotkey`, `X-Signature`, `X-Timestamp`. Message to sign: `SHA256(request_body):timestamp` (UTF-8), with your validator keypair. See the [API reference](https://docs.leoma.ai/api) in the docs.
118+
119+
---
120+
121+
## Miner setup
122+
123+
To run a **miner** on the Leoma subnet: upload your **Text-Image to Video (TI2V)** model to Hugging Face, deploy to Chutes, and commit on-chain.
124+
125+
### 1. Upload your model to Hugging Face
126+
127+
- Fine-tune or adapt a **TI2V** model, then upload it to [Hugging Face](https://huggingface.co/) as a model repository.
128+
- **Model naming (required):** The repository name must **start with `leoma`** and **end with your miner hotkey** (SS58 address).
129+
Example: `your_username/leoma-5F3sa2TJAWMqDhxG6jhV4N8ko9SxwGy8TpaNS1repo5DvT9`.
130+
- **Revision (required):** Use a **specific revision** — the full Git **commit SHA** of the model version you deploy. Do not use branch names like `main`.
131+
132+
### 2. Deploy to Chutes and commit on-chain
133+
134+
1. **Deploy to Chutes** (so validators can call your model):
135+
```bash
136+
leoma miner push --model-name <your-hf-repo> --model-revision <full-commit-sha> --chutes-api-key <api-key> --chute-user <chutes-username>
137+
```
138+
Use the **full commit SHA** as `--model-revision`. Note the **Chute ID** from the output.
139+
140+
2. **Commit on-chain** (register model + Chute for validators):
141+
```bash
142+
leoma miner commit --model-name <your-hf-repo> --model-revision <full-commit-sha> --chute-id <chute-id> --coldkey <wallet-name> --hotkey <ss58-address>
143+
```
144+
Your wallet (coldkey/hotkey) must be registered on the subnet.
145+
146+
### 3. Monitor your miner
147+
148+
- **Network page (app):** View leaderboard, valid miners, and recent evaluations. Confirm your hotkey appears and is **valid**.
149+
- **CLI:** Fetch the current rank list (same data as the dashboard):
150+
```bash
151+
leoma get-rank
152+
```
153+
- **API:** `GET /miners/list`, `GET /miners/{hotkey}`, `GET /scores/rank` — check `is_valid`, `invalid_reason`, and `eligible` (completeness ≥ 80%).
154+
155+
---
156+
157+
## Storage and API
158+
159+
- **Storage (Hippius S3):** Source videos live in the **source bucket**; task artifacts (first frame, original clip, generated videos) and evaluation results live in the **samples bucket**. Validators need **read-only** access to the samples bucket. See the [Storage](https://docs.leoma.ai/storage) doc.
160+
- **API:** The Leoma API provides health, miners, samples, scores, tasks, weights, and blacklist endpoints. Validators use **GET /tasks/latest**, **POST /samples/batch**, and **GET /weights**. See the [API reference](https://docs.leoma.ai/api).
161+
162+
---
163+
164+
## Security and production
165+
166+
- **Dependencies:** Critical packages are pinned in `pyproject.toml`. Before production, run `pip audit` (or use Dependabot/Snyk) for known vulnerabilities.
167+
- **Production env:** Set `LEOMA_ENV=production` (or `ENVIRONMENT=production`) so the API enforces non-default DB credentials and exception logs omit full tracebacks.
168+
- **CORS:** Set `CORS_ORIGINS` to a comma-separated list of allowed frontend origins. Leave unset for development (allows `*`).
169+
- **API auth:** Validator requests use hotkey signature auth; admin-only actions require hotkeys listed in `ADMIN_HOTKEYS`. See `env.example` for `SIGNATURE_EXPIRY_SECONDS` and related options.
170+
171+
---
172+
173+
## Documentation
174+
175+
Full documentation (getting started, miner setup, validator setup, storage, API reference):
176+
177+
- **Docs site:** [https://docs.leoma.ai](https://docs.leoma.ai)
178+
179+
Resources:
180+
181+
- **App / dashboard:** Leoma frontend (Overview, Product, Network, Docs, Help)
182+
- **Whitepaper:** Protocol details and incentives
183+
- **Community:** Discord, Twitter, GitHub (see the app Help page)
184+
185+
---
186+
187+
## License
188+
189+
MIT

0 commit comments

Comments
 (0)