Codebase Refactor: Types, SOLID Principles, and Architectural Separation

[joellidin]

Summary

The Crusades codebase requires a comprehensive refactor to improve maintainability, type safety, and architectural clarity. This issue tracks structural improvements following Python best practices and SOLID principles.

Scope

1. Type Safety

• Add comprehensive type hints to all public functions and classes
• Replace Any and bare dict/list with concrete generics (dict[str, Any], list[int])
• Add py.typed marker for PEP 561 compliance
• Integrate mypy --strict into CI pipeline

2. Python Idioms

• Replace manual loop index tracking with enumerate() where appropriate
• Use context managers (with statements) for resource cleanup
• Adopt pathlib over os.path for path operations
• Use dataclasses for simple data containers
• Replace manual __init__ assignments with @dataclass or attrs

3. SOLID Principles

Single Responsibility

• Split validator.py into coherent modules: blockchain interface, evaluation orchestrator, scoring logic, weight setting
• Separate Docker/Basilica execution strategies into distinct executor classes
• Extract security scanning into dedicated scanner module

Open/Closed

• Define abstract base classes for evaluation backends (AbstractExecutor)
• Use strategy pattern for MFU calculation methods
• Plugin architecture for verification checks

Liskov Substitution

• Ensure all executor implementations (Docker, Basilica, Local) are interchangeable
• Standardize result dataclasses across execution modes

Interface Segregation

• Split monolithic config objects into role-specific interfaces
• Separate miner-facing vs validator-facing APIs

Dependency Inversion

• Inject dependencies (database, blockchain client) rather than instantiating internally
• Use factories for creating execution contexts

4. DRY (Don't Repeat Yourself)

• Extract common validation logic between env.py and simulate_validator.py
• Centralize security policy definitions (single source of truth already started in security_defs.py)
• Unify MFU calculation formulas across validator and miner tooling
• Share constants between production and test environments

5. Separation of Concerns

• Domain layer: MFU calculation, scoring algorithms, verification logic
• Infrastructure layer: Docker API, Basilica client, database access, blockchain interface
• Application layer: Orchestration, commit processing, leaderboard management
• Presentation layer: TUI dashboard, logging formats

Specific Targets

Immediate

• Refactor neurons/validator.py (currently too large)
• Extract execution backends into executors/ module
• Create models/ directory for dataclasses (Submission, Evaluation, Score)
• Add type stubs for external dependencies (bittensor, etc.)

Structural

• Implement repository pattern for database access
• Create service layer for business logic
• Separate configuration from execution (pydantic settings)

Out of Scope

• Unit testing infrastructure (tracked in separate issue)
• Feature additions or algorithm changes
• Security policy modifications

Acceptance Criteria

• mypy --strict passes with zero errors
• No module exceeds 500 lines
• All public APIs have docstrings and type hints
• No circular imports introduced
• Docker and Basilica modes remain fully functional

Notes

This refactor should not change behavior—only structure. All existing tests (once added separately) must continue to pass.

[machine-god-deus]

Machine God picked this up and is drafting a spec now.

[machine-god-deus]

Machine God approved the implementation spec and has begun implementation.

Approved spec

Here is the implementation spec.

---

Implementation Spec: Codebase Refactor — Types, SOLID Principles, Architectural Separation

Overview

Structural refactor of the Crusades codebase. Three god objects (env.py 3,376 LOC, runner.py 1,519 LOC, validator.py 1,344 LOC) are decomposed into focused modules following SOLID principles. Type hints are added to all public APIs. No behavioral changes. No new features.

The codebase is a Python 3.12 Bittensor subnet with async SQLAlchemy, FastAPI, Docker/Basilica execution, and Pydantic config. Good foundations exist in protocols.py, models.py, exceptions.py, and config.py — the refactor extends these patterns outward.

Scope

1. Extract domain models — dataclasses for Submission, Evaluation, Score, MFUResult into src/crusades/models/.
2. Split database.py into DAOs — SubmissionDAO, EvaluationDAO, StateDAO, PaymentDAO with injected session factories.
3. Extract security scanner from env.py — AST scanning, forbidden-pattern checks, and security_defs.py helpers into src/crusades/security/.
4. Define AbstractExecutor and split runner.py — DockerExecutor and BasilicaExecutor implementing a common protocol in src/crusades/affinetes/executors/.
5. Decompose validator.py — extract CommitmentProcessor, SubmissionEvaluator, WeightManager as service classes.
6. Decompose env.py — extract model loading, MFU calculation, verification logic, and execution orchestration into separate modules under environments/templar/.
7. Add type hints — all public function signatures, replace bare Any/dict/list with concrete generics, add py.typed marker.
8. Convert plain classes to dataclasses — SandboxResult, ValidationResult in protocols.py and similar data holders.
9. DRY extraction — unify MFU calculation between env.py and validator tooling; centralize validation logic shared between env.py and local_test/simulate_validator.py.

Out of Scope

• Unit test infrastructure (separate issue).
• Feature additions or algorithm changes.
• Security policy modifications (content of scanning rules unchanged).
• TUI refactoring (tui/app.py, tui/client.py) — presentation layer, low priority.
• miner.py refactoring — secondary target, only touched if needed for shared extractions.
• CI pipeline changes beyond mypy config in pyproject.toml.

Files/Components Likely to Change

New Files

Path	Purpose
src/crusades/models/__init__.py	Domain dataclasses package
src/crusades/models/submission.py	Submission, SubmissionScore dataclasses
src/crusades/models/evaluation.py	EvaluationResult, MFUResult dataclasses
src/crusades/storage/submission_dao.py	Submission CRUD
src/crusades/storage/evaluation_dao.py	Evaluation CRUD
src/crusades/storage/state_dao.py	Validator state persistence
src/crusades/storage/payment_dao.py	Payment record queries
src/crusades/security/__init__.py	Security scanner package
src/crusades/security/scanner.py	AST-based code validator (extracted from env.py)
src/crusades/security/policies.py	Forbidden-pattern definitions (refactored from security_defs.py)
src/crusades/affinetes/executors/__init__.py	Executor package
src/crusades/affinetes/executors/base.py	ExecutorProtocol definition
src/crusades/affinetes/executors/docker_executor.py	Docker execution strategy
src/crusades/affinetes/executors/basilica_executor.py	Basilica execution strategy
src/crusades/services/__init__.py	Service layer package
src/crusades/services/commitment_processor.py	Commitment reading/decryption orchestration
src/crusades/services/submission_evaluator.py	Download + evaluate orchestration
environments/templar/model_loader.py	Model loading logic (from env.py)
environments/templar/mfu.py	MFU calculation (from env.py)
environments/templar/verifier.py	Weight/gradient/loss verification (from env.py)
environments/templar/execution.py	Torchrun orchestration (from env.py)
src/crusades/py.typed	PEP 561 typed marker

Modified Files

Path	Change
environments/templar/env.py	Reduced from ~3,376 to ~500 LOC. Becomes thin FastAPI shell importing from extracted modules.
src/crusades/affinetes/runner.py	Reduced from ~1,519 to ~400 LOC. Delegates to executor implementations.
neurons/validator.py	Reduced from ~1,344 to ~400 LOC. Delegates to service classes.
src/crusades/storage/database.py	Reduced from ~603 to ~150 LOC. Becomes session factory + DAO wiring.
src/crusades/core/protocols.py	Extended with ExecutorProtocol, typed StorageBackend. SandboxResult/ValidationResult converted to dataclasses.
src/crusades/core/security_defs.py	Retained as constant definitions, helper functions moved to security/scanner.py.
src/crusades/config.py	Type hint refinements only.
src/crusades/core/exceptions.py	Type hint refinements only.
pyproject.toml	Update mypy config to strict = true. Add py.typed to package-data.

Implementation Plan

Executed bottom-up. Each step is independently verifiable via ruff check + mypy + import smoke test.

Step 1: Domain Models

Extract pure dataclasses into src/crusades/models/. These have zero dependencies — safest starting point.

• Create submission.py with @dataclass versions of submission/score data.
• Create evaluation.py with @dataclass for EvaluationResult, MFUResult.
• Update protocols.py: convert SandboxResult, ValidationResult to @dataclass.

Step 2: DAO Extraction

Split database.py into focused DAOs. Each receives async_sessionmaker via constructor.

• SubmissionDAO: save, get, update_status, get_pending, get_leaderboard, get_top.
• EvaluationDAO: save, get_by_submission.
• StateDAO: get/update validator state, adaptive threshold.
• PaymentDAO: save/get verified payments.
• database.py becomes a facade: holds session factory, exposes DAOs, preserves the existing Database public API for backward compat during transition.

Step 3: Security Scanner Extraction

Extract ~1,000 lines of AST scanning from env.py into src/crusades/security/.

• scanner.py: CodeScanner class implementing CodeValidator protocol.
• policies.py: re-exports constants from security_defs.py with type annotations.
• env.py imports CodeScanner and calls scanner.validate(code).
• security_defs.py remains as the single source of truth for forbidden patterns.

Step 4: Executor Abstraction

Split runner.py into strategy-pattern executors.

• Define ExecutorProtocol in executors/base.py (matches existing SandboxRuntime protocol, extended with config).
• DockerExecutor: all Docker-specific logic from runner.py.
• BasilicaExecutor: all Basilica-specific logic + BasilicaDeploymentContext.
• AffinetesRunner becomes an orchestrator: selects executor based on config, delegates run_evaluation().

Step 5: Validator Decomposition

Extract service classes from neurons/validator.py.

• CommitmentProcessor: reads blockchain commitments, filters by spec version, deduplicates.
• SubmissionEvaluator: downloads code from URL, validates, triggers evaluation, records result.
• Weight setting remains in chain/weights.py (already well-separated).
• Validator class becomes a thin orchestration loop: run_step() calls services in sequence.

Step 6: env.py Decomposition

Split the evaluation environment into focused modules under environments/templar/.

• model_loader.py: _load_model(), weight initialization, config resolution.
• mfu.py: _calculate_mfu(), GPU FLOP constants, parallelism detection.
• verifier.py: _verify_trainable_params(), _verify_params_changed(), _verify_final_weights(), _verify_outputs().
• execution.py: _evaluate_via_torchrun(), subprocess management.
• env.py retains: FastAPI app, /evaluate endpoint, Actor class as thin coordinator.

Step 7: Type Hints + mypy --strict

• Add type annotations to all public functions in new and modified modules.
• Replace Any in StorageBackend protocol with concrete domain types.
• Add py.typed marker to src/crusades/.
• Create src/crusades/stubs/ with minimal type stubs for bittensor (untyped external dep).
• Update pyproject.toml: set mypy strict = true, add per-module overrides for third-party code.

Step 8: DRY Consolidation

• Unify MFU formula: environments/templar/mfu.py becomes the canonical implementation. local_test/simulate_validator.py imports from it (or from a shared src/crusades/core/mfu.py if the env isolation boundary requires duplication).
• Centralize security policy: security_defs.py remains single source. Remove any inline copies in env.py.

Test and Verification Plan

Verification Contract

The provided verification contract (docker compose -f docker/compose.yml up -d) is misaligned with this task. The compose file starts a full validator node requiring NVIDIA GPUs, bittensor wallets, and a running blockchain network. A structural refactor cannot be verified this way.

Required verification override:

# .machine-god/verification.yaml
boot: "pip install -e '.[dev]'"
readiness: "python -c 'import crusades'"
integration:
  - "ruff check src/ neurons/ environments/"
  - "mypy --strict src/crusades/ neurons/"
  - "python -c 'from crusades.affinetes.runner import AffinetesRunner'"
  - "python -c 'from crusades.security.scanner import CodeScanner'"
  - "python -c 'from crusades.storage.database import Database'"
  - "python -c 'from neurons.validator import Validator'"
  - "python -c 'from environments.templar.env import app'"
teardown: "echo 'no teardown needed'"

Manual Verification Steps

1. ruff check --select E,F,I,N,W,UP src/ neurons/ environments/ — zero errors.
2. mypy --strict src/crusades/ — zero errors (third-party ignore overrides permitted).
3. Import smoke tests for every refactored module — no ImportError, no circular imports.
4. python -c "from crusades.core.protocols import SandboxRuntime, CodeValidator, StorageBackend" — protocols importable.
5. Grep for removed functions/classes — verify all references updated, no dangling imports.

Guardrail Exceptions

File/Directory	Reason
pyproject.toml	Must update mypy config to strict = true and add py.typed to package-data. No dependency changes.
environments/templar/Dockerfile	May need to COPY new module files (model_loader.py, mfu.py, verifier.py, execution.py) alongside env.py. The Dockerfile currently copies individual files, not a directory.
src/crusades/core/security_defs.py	Type annotations added to helper functions. No policy content changes. Listed because it is security-critical code.

No CI workflow files exist in the repo (.github/ contains workflows but they are not modified by this task). No lint config files are changed beyond pyproject.toml mypy settings.

Parallel Work Units

Unit 1: Domain Models + DAO Extraction

Steps 1–2. Pure data layer, no upstream dependencies.
Files:

• src/crusades/models/__init__.py, submission.py, evaluation.py
• src/crusades/storage/submission_dao.py, evaluation_dao.py, state_dao.py, payment_dao.py
• src/crusades/storage/database.py (modified)
• src/crusades/core/protocols.py (dataclass conversions)

Unit 2: Security Scanner Extraction

Step 3. Depends only on security_defs.py constants.
Files:

• src/crusades/security/__init__.py, scanner.py, policies.py
• src/crusades/core/security_defs.py (type hints only)

Unit 3: Executor Abstraction

Step 4. Depends on domain models from Unit 1.
Files:

• src/crusades/affinetes/executors/__init__.py, base.py, docker_executor.py, basilica_executor.py
• src/crusades/affinetes/runner.py (reduced)

Unit 4: Validator + env.py Decomposition

Steps 5–6. Depends on Units 1–3 for types and executor interfaces.
Files:

• src/crusades/services/__init__.py, commitment_processor.py, submission_evaluator.py
• neurons/validator.py (reduced)
• environments/templar/model_loader.py, mfu.py, verifier.py, execution.py
• environments/templar/env.py (reduced)
• environments/templar/Dockerfile (COPY updates)

Unit 5: Type Hints + DRY

Steps 7–8. Runs last, touches all modules.
Files:

• All public APIs across src/crusades/, neurons/, environments/templar/
• src/crusades/py.typed
• pyproject.toml (mypy strict)

Shared/Coordination (main agent first)

• src/crusades/core/protocols.py — extended in Unit 1, consumed by all others.
• src/crusades/config.py — referenced by Units 3 and 4.
• pyproject.toml — modified in Unit 5 only.

Dependency graph: Unit 1 and Unit 2 are fully parallel. Unit 3 follows Unit 1. Unit 4 follows Units 1–3. Unit 5 follows all.

Risks / Open Questions

1. No test suite. Every extraction is a behavior-preservation bet verified only by import smoke tests and linting. Highest risk factor. Mitigation: small, mechanical moves; preserve all function signatures; keep original files as thin wrappers during transition.

2. env.py runs inside containers. The evaluation Dockerfile copies env.py and security_defs.py by name. Splitting env.py into modules requires updating the Dockerfile COPY directives. If the Basilica image build is separate or cached, this must be coordinated.

3. Bittensor is untyped. mypy --strict will require either stub files or per-import # type: ignore[import-untyped] annotations. Creating full stubs for bittensor is out of scope — minimal stubs covering actually-used APIs is the pragmatic path.

4. Global singletons in config.py and database.py. The @cache / module-level _config pattern is load-bearing. Refactoring to DI is desirable but risks breaking import-time initialization. Plan: preserve singletons as the default path, add DI-capable constructors alongside.

5. Circular import risk. Extracting CommitmentProcessor from validator.py requires importing from chain/commitments.py, storage/, and config.py. Must ensure no module in chain/ or storage/ imports from services/ or neurons/.

6. security_defs.py is duplicated into the eval container. The Dockerfile copies it independently. If security/scanner.py imports from security_defs.py, the container must either copy the full security/ package or env.py must inline its own scanner call. The eval container runs env.py standalone — it does not install the crusades package. This isolation boundary constrains how much of env.py can reference src/crusades/ modules.

7. MFU formula unification. env.py (inside container) and local_test/simulate_validator.py (outside container) both compute MFU. Because env.py runs in an isolated container without the crusades package, true code sharing requires either: (a) copying the shared module into the container, or (b) accepting the duplication with a comment linking the canonical source. Option (a) is preferred.

[machine-god-deus]

Machine God completed delivery.

Pull requests:

• Codebase Refactor: Types, SOLID Principles, and Architectural Separation #82