Fetching API docs occasionally fails at starlette framework middleware issue with streaming large responses

[alidaw]

Fetching the docs of API built with FastAPI framework occasionally fails at the point where schema gets processed and the property called attributes as expected by JSON:API is missing.
From what the error traceback taken from running container reveals in code snippet shown below is that the root error isn’t caused by mentioned attributes KeyError, but the traceback ends in attributes KeyError as kind of “domino effect” caused by a async memory stream issue.

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 98, in receive
    return self.receive_nowait()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 93, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 78, in call_next
    message = await recv_stream.receive()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/anyio/streams/memory.py", line 118, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 428, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 290, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 169, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 167, in __call__
    await self.app(scope, receive, send_wrapper)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/app/middleware.py", line 96, in dispatch
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/app/middleware.py", line 66, in dispatch
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/app/middleware.py", line 46, in dispatch
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 245, in openapi
    return JSONResponse(self.openapi())
                        ^^^^^^^^^^^^^^
  File "/code/app/main.py", line 99, in custom_openapi
    app.openapi_schema = schemas.get_schema(app.routes)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette_jsonapi/openapi.py", line 253, in get_schema
    self.spec.path(path=new_path, operations={endpoint.http_method: openapi_info}, parameters=path_params)
  File "/usr/local/lib/python3.11/site-packages/apispec/core.py", line 534, in path
    plugin.operation_helper(path=path, operations=operations, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/__init__.py", line 224, in operation_helper
    self.resolver.resolve_operations(operations)
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/schema_resolver.py", line 33, in resolve_operations
    self.resolve_schema(operation["requestBody"])
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/schema_resolver.py", line 231, in resolve_schema
    content["schema"] = self.resolve_schema_dict(content["schema"])
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/schema_resolver.py", line 295, in resolve_schema_dict
    return self.converter.resolve_nested_schema(schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/openapi.py", line 134, in resolve_nested_schema
    self.spec.components.schema(name, schema=schema)
  File "/usr/local/lib/python3.11/site-packages/apispec/core.py", line 171, in schema
    ret.update(plugin.schema_helper(component_id, ret, **kwargs) or {})
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/__init__.py", line 182, in schema_helper
    json_schema = self.converter.schema2jsonschema(schema_instance)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/apispec/ext/marshmallow/openapi.py", line 253, in schema2jsonschema
    jsonschema = self.fields2jsonschema(fields, partial=partial)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/starlette_jsonapi/openapi.py", line 116, in fields2jsonschema
    properties['attributes'].setdefault('required', []).append(observed_field_name)
    ~~~~~~~~~~^^^^^^^^^^^^^^
KeyError: 'attributes'

The issue seems to be related to an issue promised to have been fixed with starlette release 0.13.7 (Fix high memory usage when using BaseHTTPMiddleware middleware classes and streaming responses), but the issue still seems to be an issue. See also issue raised on GitHub @ Memory usage streaming large responses · Issue #1012 · encode/starlette.

As it is annoying to occasionally face the mentioned issue when simply fetching the API docs it would be great if you could fix it.
Fetching of API docs usually shouldn't be an issue. As it is a server sided issue with HTTP 500 status getting responded it degrades the quality of my API. I didn't faced that issue using other frameworks for building APIs.

Important

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[RandyRomero]

They seem to have reverted this fix later on in release 0.13.8 (they also mentioned it here). At the same time, they said it should have been fixed with this fix in 0.15.0

[RandyRomero]

@alidaw was wondering what if it is not a async memory stream issue, but actual KeyError in starlette-jsonapi ?

I was looking at the code where this is happening and it seems there could be a logic problem:

if a field is called ‘id’, it might not get this attributes attribute assigned. At the same time it would be accessed if this field is required.

I doubt this is an actual problem, because it would be too obvious and many people would get the same problem here. However, it might explain the error we have, so I will try to look deeper into it

[alidaw]

Yes. You are right. Seems as if parameter called fields reaches starlette_jsonapi.openapi.JSONAPISchemaConverter.fields2jsonschema() method with randomly ordered keys and the fail in logic applied to the iteration over the keys the parameter called fields of data type dictionary bears doesn't consider running into KeyError when accessing key called attributes.