Skip to content

Latest commit

 

History

History
81 lines (69 loc) · 7.46 KB

File metadata and controls

81 lines (69 loc) · 7.46 KB

Distributed platform validation

Validation date: 2026-07-24. Scope: controlled local infrastructure only. These results are not external-project validation, cloud-deployment evidence, production scale, adoption, or time saved. Raw evidence is retained in validation/platform/results-strict.json, results-benchmark.json, and results-infrastructure.json.

Requirement ledger

Requirement Status Executed evidence / limitation
Signed webhook through completed replay passed Real HMAC webhook, PostgreSQL tenant/event, encrypted MinIO artifact, durable queue, Docker worker, and Python replay completed in one attempt.
Tenant isolation and duplicate delivery/request handling passed Cross-tenant read returned 404; duplicate webhook returned the recorded duplicate response; duplicate replay idempotency key returned 409.
Artifact integrity and signatures passed Valid encrypted object replayed; incorrect upload digest returned 422; overwritten object failed verification on every attempt.
Cause-preserving failure classification passed A real Docker API permission failure returned exit code 1 against an artifact expecting exit code 1 and was correctly classified reproduced: false; a regression test prevents infrastructure text from matching a test failure.
Secret redaction passed A known GITHUB_TOKEN canary was absent from every archived member.
Retries, backoff, corruption, and DLQ passed Corrupted MinIO object made five real attempts and reached dead_letter in 30.442 seconds with artifact digest mismatch.
Lease expiry, worker crash, checkpoint recovery passed Worker was killed during a five-second replay; the expired lease was reclaimed and checkpoint advanced from worker_claimed to completed on attempt 2 in 5.383 seconds.
PostgreSQL restart recovery passed PostgreSQL was restarted; readiness, webhook intake, upload, queueing, and replay recovered in 1.477 seconds.
Prometheus observability passed Real control-plane and worker endpoints were scraped; the final worker replay incremented reproci_worker_success_total and execution latency.
OpenTelemetry export passed Official Collector Contrib 0.130.1 received one resource-span batch containing three spans from a real smoke replay. No hosted backend claim.
Retention controls failed MinIO lifecycle export confirmed the one-day expiration rule, but actual deletion was not observed because the 24-hour boundary was not awaited. Configuration alone is not execution evidence.
Strict integration mode passed REPROCI_VALIDATION_REQUIRED=1 go test -tags=integration ./... used real PostgreSQL and MinIO; missing dependency variables call t.Fatal, never t.Skip.
Controlled load at 1, 10, 25, and 50 concurrent jobs passed Two complete runs per level; 172/172 controlled jobs reproduced. Raw per-job records are retained.
Local Kubernetes passed Kind v0.31.0/Kubernetes 1.34.3: two control-plane replicas Ready and a numeric-non-root replay Job completed with the expected signature.
Terraform passed Official Terraform 1.15.8 container, AWS provider 5.100.0: format, init, and validate completed with zero errors and zero warnings. No plan/apply or cloud claim.
GitHub App installation and real deliveries passed for owner-controlled dogfood Least-privilege App 4386569 was installed on exactly two owner repositories; 36/36 expected signed workflow_job deliveries were accepted after final configuration. The temporary webhook is now disabled.
AWS plan/apply and hosted observability externally pending Requires AWS credentials, budget, DNS/TLS decisions, and deployment approval.
Two-repository owner dogfood passed with qualification Four public workflow runs across two owner-controlled repositories produced 12/12 controlled replays. This is not independent adoption or flaky-test validation.
Independent beta users externally pending No person was contacted and no independent repository was modified.

Controlled benchmark

Each level ran twice against the real Compose stack. Reproducibility was 100% for this single controlled artifact only.

Concurrent jobs Throughput jobs/s Queue p95 (s) Completion p95 (s)
1 3.557 / 3.614 0.031 / 0.046 0.223 / 0.229
10 13.873 / 17.548 0.071 / 0.031 0.680 / 0.553
25 18.342 / 18.093 0.121 / 0.085 1.309 / 1.323
50 17.640 / 18.501 0.208 / 0.149 2.781 / 2.638

The 1,316-byte plaintext artifact occupied 1,332 bytes after AES-256-GCM encryption: 16 bytes of object-data overhead. The replays PostgreSQL relation grew 32,768 bytes during the retained 172-job run. Sampled maximum single-container memory was 386,400,256 bytes; sampled maximum single-container CPU was 16.79%. These are Docker Desktop observations, not host-wide resource totals or capacity claims.

Exact resume-bullet phrase ledger

Exact phrase Status Permitted interpretation
“Built an open-source system” passed Implemented repository and locally executed platform.
“captures failed CI jobs” passed The released Action captured controlled failures in 12 successful public GitHub Actions trials, in addition to local validation.
“deterministically replays them in isolated containers” passed Docker and Kind replays reproduced controlled artifacts; 12/12 post-adaptation owner-dogfood artifacts reproduced in one attempt. No cross-project generalization claim.
“helping developers reproduce intermittent test failures” externally pending Product intent only; no external user or debugging-time study.
“distributed execution” passed PostgreSQL queue and 50-worker local execution were exercised.
“content-addressed caching” passed Existing SHA-256 cache is preserved; hosted artifacts are digest-verified.
“sandboxing” passed for declared controls Read-only filesystem, non-root UID, dropped capabilities, no-new-privileges, resource limits, and no-network Docker replay were exercised; not a complete security boundary.
“checkpointing” passed A killed worker resumed from an expired lease and completed with a persisted checkpoint.
“filesystem/network fault injection” passed Supported only by the existing controlled engine benchmark; not evidence of external failure coverage.
“replay 50 concurrent jobs with 100% reproducibility” passed Two 50-job controlled runs completed 50/50; do not quote as a real-world reproduction rate.
“Piloted with [N] open-source projects” externally pending Two public owner repositories were dogfooded, but that is not an independent pilot or adoption.
“reduced median failure-reproduction time by [X]%” externally pending No user baseline or time study exists.
“leading to [N] resolved flaky tests and [N] external contributions” externally pending No external repository was modified and nobody was contacted.

Strict reproduction commands

docker compose -f compose.platform.yaml up -d --build
python validation/platform/strict_validation.py
python validation/platform/benchmark_platform.py

cd controlplane
REPROCI_VALIDATION_REQUIRED=1 \
DATABASE_URL=postgres://reproci:local-only-change-me@127.0.0.1:5432/reproci \
REPROCI_S3_BUCKET=reproci REPROCI_S3_ENDPOINT=http://127.0.0.1:9000 \
AWS_ACCESS_KEY_ID=local-reproci AWS_SECRET_ACCESS_KEY=local-only-change-me \
AWS_REGION=us-east-1 go test -tags=integration ./... -count=1