fix: resolve test stalling and add MCP schema validation#21
Merged
crowecawcaw merged 2 commits intomainfrom Jan 12, 2026
Merged
fix: resolve test stalling and add MCP schema validation#21crowecawcaw merged 2 commits intomainfrom
crowecawcaw merged 2 commits intomainfrom
Conversation
This commit addresses two critical issues and adds comprehensive schema validation: 1. Fixed test stalling issue caused by pydantic serialization error - Root cause: Click's Sentinel.UNSET values were being included in tool schemas - Solution: Filter out non-JSON-serializable defaults (Sentinel, enums, etc.) - Tests now complete in ~37s instead of timing out after 60s+ 2. Fixed enum support for click.Choice parameters - Enums are now properly extracted from param.type.choices - Enables proper validation of choice-based parameters 3. Added pytest timeout configuration - Set default timeout to 10s per test - Added pytest-timeout to test dependencies - Prevents future test hangs from blocking CI/CD 4. Integrated official MCP JSON Schema (2025-06-18) - Downloaded and saved official schema to tests/schemas/ - All tool definitions now validate against official spec 5. Created comprehensive schema validation test suite (15 new tests) - Validates all generated tools against official MCP schema - Tests basic, complex, and edge case scenarios - Covers real CLI fixtures (basic_cli, advanced_cli, context_cli) - Ensures JSON Schema draft 2020-12 compliance - Added jsonschema dependency for validation All 74 tests now pass successfully (59 existing + 15 new).
The MCP schema validates basic structure of inputSchema but does NOT validate that the inputSchema itself is a valid JSON Schema. This commit adds comprehensive validation to ensure inputSchema compliance with JSON Schema draft 2020-12. Key additions: - validate_input_schema_is_valid_json_schema() function using Draft202012Validator.check_schema() - 7 new test cases specifically validating JSON Schema compliance: * All inputSchemas are valid JSON Schema draft 2020-12 * Type field is present and set to "object" * All properties have required "type" field * Enum properties are valid JSON Schema * Required array references valid properties * Real fixtures produce valid JSON Schemas * Default values match their declared types This addresses the gap where MCP schema uses "additionalProperties: true" for property schemas, allowing invalid JSON Schema to pass. All 81 tests pass (59 existing + 22 schema validation tests).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit addresses two critical issues and adds comprehensive schema validation:
Fixed test stalling issue caused by pydantic serialization error
Fixed enum support for click.Choice parameters
Added pytest timeout configuration
Integrated official MCP JSON Schema (2025-06-18)
Created comprehensive schema validation test suite (15 new tests)
All 74 tests now pass successfully (59 existing + 15 new).