Skip to content

Commit 7584e70

Browse files
omoshola-oomoshola-o
committed
feat: add validation suite with 61 invalid examples and CI pipeline
- 61 invalid examples across all 7 schemas testing missing required fields, bad enum values, pattern violations, type mismatches, range violations, and conditional constraint failures - Updated validate_schemas.py with parametrized pytest tests for both valid (18) and invalid (61) examples - Standalone runner: python -m xap.tests.validate_schemas - GitHub Actions CI: validates on every push/PR to main - 101/101 tests pass (22 existing + 79 schema validation)
1 parent 8969b01 commit 7584e70

64 files changed

Lines changed: 1049 additions & 8 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/validate.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name: XAP Schema Validation
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
validate:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- uses: actions/checkout@v4
14+
- uses: actions/setup-python@v5
15+
with:
16+
python-version: '3.11'
17+
- name: Install dependencies
18+
run: pip install jsonschema pytest
19+
- name: Run schema validation
20+
run: pytest xap/tests/ -v

‎pytest.ini‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
[pytest]
22
pythonpath = .
3+
python_files = test_*.py validate_schemas.py

‎xap/tests/invalid/agent-identity/bad-agent-id-format.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"agent_id": "agent_ZZZZZZZZ",
3+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
4+
"key_version": 1,
5+
"key_status": "active",
6+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 300, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
7+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
8+
"xap_version": "0.2.0",
9+
"status": "active",
10+
"registered_at": "2026-03-09T12:00:00Z",
11+
"last_active_at": "2026-03-09T12:00:00Z",
12+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
13+
}

‎xap/tests/invalid/agent-identity/bad-reputation-bps.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"agent_id": "agent_7f3a9b2c",
3+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
4+
"key_version": 1,
5+
"key_status": "active",
6+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 300, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
7+
"reputation": {"total_settlements": 0, "success_rate_bps": 15000, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
8+
"xap_version": "0.2.0",
9+
"status": "active",
10+
"registered_at": "2026-03-09T12:00:00Z",
11+
"last_active_at": "2026-03-09T12:00:00Z",
12+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
13+
}

‎xap/tests/invalid/agent-identity/bad-status.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"agent_id": "agent_7f3a9b2c",
3+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
4+
"key_version": 1,
5+
"key_status": "active",
6+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 300, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
7+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
8+
"xap_version": "0.2.0",
9+
"status": "deleted",
10+
"registered_at": "2026-03-09T12:00:00Z",
11+
"last_active_at": "2026-03-09T12:00:00Z",
12+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
13+
}

‎xap/tests/invalid/agent-identity/empty-capabilities.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"agent_id": "agent_7f3a9b2c",
3+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
4+
"key_version": 1,
5+
"key_status": "active",
6+
"capabilities": [],
7+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
8+
"xap_version": "0.2.0",
9+
"status": "active",
10+
"registered_at": "2026-03-09T12:00:00Z",
11+
"last_active_at": "2026-03-09T12:00:00Z",
12+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
13+
}

‎xap/tests/invalid/agent-identity/extra-field.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"agent_id": "agent_7f3a9b2c",
3+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
4+
"key_version": "not_an_integer",
5+
"key_status": "active",
6+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 300, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
7+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
8+
"xap_version": "0.2.0",
9+
"status": "active",
10+
"registered_at": "2026-03-09T12:00:00Z",
11+
"last_active_at": "2026-03-09T12:00:00Z",
12+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
13+
}

‎xap/tests/invalid/agent-identity/float-pricing.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"agent_id": "agent_7f3a9b2c",
3+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
4+
"key_version": 1,
5+
"key_status": "active",
6+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 3.50, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
7+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
8+
"xap_version": "0.2.0",
9+
"status": "active",
10+
"registered_at": "2026-03-09T12:00:00Z",
11+
"last_active_at": "2026-03-09T12:00:00Z",
12+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
13+
}

‎xap/tests/invalid/agent-identity/missing-agent-id.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"public_key": "ed25519:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=",
3+
"key_version": 1,
4+
"key_status": "active",
5+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 300, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
6+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
7+
"xap_version": "0.2.0",
8+
"status": "active",
9+
"registered_at": "2026-03-09T12:00:00Z",
10+
"last_active_at": "2026-03-09T12:00:00Z",
11+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
12+
}

‎xap/tests/invalid/agent-identity/missing-public-key.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"agent_id": "agent_7f3a9b2c",
3+
"key_version": 1,
4+
"key_status": "active",
5+
"capabilities": [{"name": "text_summarization", "version": "1.0.0", "description": "Summarizes text", "pricing": {"model": "fixed", "amount_minor_units": 300, "currency": "USD", "per": "request"}, "sla": {"max_latency_ms": 3000, "availability_bps": 9900, "min_quality_score_bps": 8500}}],
6+
"reputation": {"total_settlements": 0, "success_rate_bps": 0, "disputes": 0, "dispute_resolution_rate_bps": 0, "last_updated": "2026-03-09T12:00:00Z"},
7+
"xap_version": "0.2.0",
8+
"status": "active",
9+
"registered_at": "2026-03-09T12:00:00Z",
10+
"last_active_at": "2026-03-09T12:00:00Z",
11+
"signature": "ed25519:c2lnbmF0dXJlX2V4YW1wbGVfYmFzZTY0X2VuY29kZWQ="
12+
}

0 commit comments

Comments
 (0)