How do validate against a single file containing multiple schemas?

[mdavis-xyz]

I have an OpenAPI spec. Here's a minimal example:

swagger: '2.0'
info:
  version: 1.0.0
  title: Example API
paths:
  /items:
    get:
      parameters:
      - in: query
        name: itemnum
        required: true
        type: integer
      responses:
        200:
          description: good
          schema:
            $ref: '#/definitions/Item'
        400:
          description: bad input parameter
definitions:
  Item:
    type: object
    required:
    - myThing
    properties:
      myThing:
        $ref: '#/definitions/MyThing'
  MyThing:
    type: object
    required:
      - mysubthing
    properties:
      mysubthing:
        type: number
        

I have a python dictionary, like:

{
    'myThing': {
        'mysubthing': 1
    }
}

How can I validate that this dictionary is an instance of components.schemas.Item?

Attempts

Attempt 1

import yaml
import jsonschema

with open('spec.yaml', 'r') as f:
    raw_schema = yaml.safe_load(f)
schema = raw_schema['definitions']['Item']

data = {
    'myThing': {
        'mysubthing': 1
    }
}

jsonschema.validate(data, schema)

Expected behavior: Script passes

Actual behavior:

Traceback (most recent call last):
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 948, in resolve_fragment
    document = document[part]
KeyError: 'definitions'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 14, in <module>
    jsonschema.validate(data, schema)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 1101, in validate
    error = exceptions.best_match(validator.iter_errors(instance))
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/exceptions.py", line 385, in best_match
    best = max(itertools.chain([best], errors), key=key)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 277, in iter_errors
    for error in errors:
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/_validators.py", line 332, in properties
    yield from validator.descend(
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 294, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 277, in iter_errors
    for error in errors:
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/_validators.py", line 294, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 880, in resolve
    return url, self._remote_cache(url)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 898, in resolve_from_url
    return self.resolve_fragment(document, fragment)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 950, in resolve_fragment
    raise exceptions.RefResolutionError(
jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'definitions/MyThing'

So it's stuck on resolving the reference. Fair enough, since the schema I passed was only the definitions/Item subset, which does not contain definitions/MyThing.

Attempt 2

import yaml
import jsonschema

with open('spec.yaml', 'r') as f:
    raw_schema = yaml.safe_load(f)
schema = raw_schema

data = {
    'myThing': {
        'mysubthing': 1
    }
}

jsonschema.validate(data, schema)

expected behavior: script runs without throwing exception

actual behavior: script runs without throwing exception

But hang on, how did jsonschema know which of the two schemas in the file to validate against?
Let's check that it can detect an invalid schema.

Attempt 3

import yaml
import jsonschema

with open('spec.yaml', 'r') as f:
    raw_schema = yaml.safe_load(f)
schema = raw_schema

good_data = {
    'myThing': {
        'mysubthing': 1
    }
}

bad_data = {
    'myThing': {
        'mysubthing': 'abc'
    }
}

jsonschema.validate(good_data, schema)

try:
    jsonschema.validate(bad_data, schema)
except jsonschema.ValidationError:
    pass
else:
    raise ValueError("Should have thrown ValidationError for invalid input")

expected behavior: Script runs without throwing an exception.

Actual behavior:

Traceback (most recent call last):
  File "main.py", line 27, in <module>
    raise ValueError("Should have thrown ValidationError for invalid input")
ValueError: Should have thrown ValidationError for invalid input

So it's not detecting invalid schemas. It's just saying everything is valid.

Next steps

So now I'm stuck. This is probably related to resolvers. The documentation for validate() does not mention a resolver argument, even though examples elsewhere in the docs use it. How am I supposed to know that resolvers is a valid kwargforvalidate()`?

Anyway, let's try to pass a RefResolver via the hidden resolver kwarg.

As mentioned in discussion 1044, I don't know what the second argument to RefResolver should be. I'm also not sure what base_uri should be, since I loaded the spec from disk into a dictionary myself. I'm not expecting the library to read anything from disk.

Attempt 4

import yaml
import jsonschema

fname = 'spec.yaml'
base_uri = f"file://{fname}"
with open(fname, 'r') as f:
    raw_schema = yaml.safe_load(f)
schema = raw_schema

good_data = {
    'myThing': {
        'mysubthing': 1
    }
}

bad_data = {
    'myThing': {
        'mysubthing': 'abc'
    }
}

resolver = jsonschema.validators.RefResolver(base_uri, True)

jsonschema.validate(good_data, schema['definitions']['Item'], resolver=resolver)

try:
    jsonschema.validate(bad_data, schema['definitions']['Item'], resolver=resolver)
except jsonschema.ValidationError:
    pass
else:
    raise ValueError("Should have thrown ValidationError for invalid input")

Same result:

Traceback (most recent call last):
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 948, in resolve_fragment
    document = document[part]
TypeError: 'bool' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 24, in <module>
    jsonschema.validate(good_data, schema['definitions']['Item'], resolver=resolver)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 1101, in validate
    error = exceptions.best_match(validator.iter_errors(instance))
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/exceptions.py", line 385, in best_match
    best = max(itertools.chain([best], errors), key=key)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 277, in iter_errors
    for error in errors:
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/_validators.py", line 332, in properties
    yield from validator.descend(
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 294, in descend
    for error in self.evolve(schema=schema).iter_errors(instance):
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 277, in iter_errors
    for error in errors:
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/_validators.py", line 294, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 880, in resolve
    return url, self._remote_cache(url)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 898, in resolve_from_url
    return self.resolve_fragment(document, fragment)
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 950, in resolve_fragment
    raise exceptions.RefResolutionError(
jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'definitions/MyThing'

Hmm, that top half of the error is interesting:

TypeError: 'bool' object is not subscriptable

This suggests to me that the FAQ is possibly wrong, and referrer should not be a bool?

Attempt 5

Change

resolver = jsonschema.validators.RefResolver(base_uri, base_uri)

to

resolver = jsonschema.validators.RefResolver(base_uri, base_uri)

Same traceback, except instead of being about bools, it says:

Traceback (most recent call last):
  File "/home/ec2-user/.pyenv/versions/3.8.11/lib/python3.8/site-packages/jsonschema/validators.py", line 948, in resolve_fragment
    document = document[part]
TypeError: string indices must be integers

Conclusion

I'm lost. I can't figure out how to use this library to validate a dictionary matches a definition inside a single OpenAPI yaml file, with internal references.

Can someone please explain to me how to do this?

[mdavis-xyz]

Oh I think I found the answer. Using RefResolver.from_schema!

This was not obvious. Perhaps we can add an FAQ, or some general documentation about this?

import yaml
import jsonschema
from jsonschema import RefResolver, Draft7Validator

fname = 'spec.yaml'
base_uri = f"file://{fname}"
with open(fname, 'r') as f:
    raw_schema = yaml.safe_load(f)
schema = raw_schema

good_data = {
    'myThing': {
        'mysubthing': 1
    }
}

bad_data = {
    'myThing': {
        'mysubthing': 'abc'
    }
}

resolver = jsonschema.validators.RefResolver.from_schema(raw_schema)

jsonschema.validate(good_data, schema['definitions']['Item'], resolver=resolver)

try:
    jsonschema.validate(bad_data, schema['definitions']['Item'], resolver=resolver)
except jsonschema.ValidationError:
    pass
else:
    raise ValueError("Should have thrown ValidationError for invalid input")

[stefanofusai]

Hey @mdavis-xyz, did you manage to find a way to migrate this code snippet to the newer referencing approach? I'm currently stuck :/

[mdavis-xyz]

No, I haven't seen this approach before. I haven't touched this stuff in over a year, sorry.