Aiven-Open / Aiven-Open/karapace
RecursionError on self-referential JSON Schema during compatibility check
- Dominant language
- Python
- Stars
- 634
- Forks
- 110
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 4
Description
# What happened?
Registering (or checking compatibility of) a **self-referential JSON Schema** — a standard, valid pattern for recursive structures like trees, linked lists, or nested comments, e.g.:
```json
{
"type": "object",
"properties": {
"name": {"type": "string"},
"children": {"type": "array", "items": {"$ref": "#"}}
}
}
```
raises an unhandled `RecursionError` instead of returning a compatibility result.
`normalize_schema_rec()` (`src/karapace/core/compatibility/jsonschema/utils.py:22`) resolves every `$ref` it meets via `resolver.resolve(ref)` and recurses into the result, with no tracking of already-visited scopes. A schema that refers to itself (directly, or through a cycle of mutual references) therefore recurses without bound.
This sits on the schema registry's **primary write path**, not just an edge-case endpoint:
- `SchemaRegistry.write_new_schema_local()` (`schema_registry.py:304`) → `check_schema_compatibility()` (`schema_registry.py:461`) → `SchemaCompatibility.check_compatibility()` (`compatibility/schema_compatibility.py:30`) → for `SchemaType.JSONSCHEMA` → `jsonschema_compatibility()` → `compatibility()` → `normalize_schema()` → `normalize_schema_rec()`.
- Also reachable directly via `Controller.compatibility_check()` (`api/controller.py:145`).
So: any `POST /subjects/{subject}/versions` for a subject that already has a live version, under any compatibility mode other than `NONE` (the Karapace default is `BACKWARD`), runs the incoming schema through this code.
Nothing catches the `RecursionError` before it reaches the HTTP layer. `setup_exception_handlers()` only registers handlers for `StarletteHTTPException` and `RequestValidationError`, so this falls through to Starlette's default handling — an unstructured `500` instead of Karapace's normal `{"error_code": ..., "message": ...}` error body.
**Minimal reproduction** (no server needed — calls the real `normalize_schema()` directly with a real `jsonschema.Draft7Validator`):
```python
import sys
from jsonschema import Draft7Validator
from karapace.core.compatibility.jsonschema.utils import normalize_schema
recursive_schema = {
"type": "object",
"properties": {
"name": {"type": "string"},
"children": {
"type": "array",
"items": {"$ref": "#"},
},
},
}
validator = Draft7Validator(recursive_schema)
print("Python recursion limit:", sys.getrecursionlimit())
print("Trying to normalize a self-referential schema...")
try:
result = normalize_schema(validator)
print("Did NOT crash. Result (truncated):", str(result)[:300])
except RecursionError as e:
print("CRASHED: RecursionError -", e)
except Exception as e:
print(f"CRASHED differently: {type(e).__name__}: {e}")
```
Output:
```
Python recursion limit: 1000
Trying to normalize a self-referential schema...
CRASHED: RecursionError - maximum recursion depth exceeded
```
# What did you expect to happen?
Either:
1. The compatibility checker treats the recursive structure correctly by tracking visited `(scope, ref)` pairs and short-circuiting on repeat (the usual way JSON Schema tooling handles self-reference), so registration succeeds or fails based on a proper bounded comparison; or
2. At minimum, the registration/compatibility-check endpoints catch this failure mode and return a structured `422`/`409` explaining the schema couldn't be compared, rather than an unhandled `RecursionError` surfacing as a generic `500`.
# What else do we need to know?
- karapace, commit `3bbc1f8d30d7b650f49b75128ba9a46fdab61a67` (2026-06-23), Python 3.12
- I only traced the JSON Schema path in detail — haven't checked whether the Avro or Protobuf compatibility checkers (dispatched separately in `schema_compatibility.py`) have the same gap, or handle self-reference correctly via their own libraries.
- Didn't spin up a full ASGI server to confirm the exact HTTP-layer behavior end-to-end; the repro exercises the function that the registration path actually calls, which is where the exception originates.
- Happy to open a PR: track a `seen: set[str]` of resolved scopes through `normalize_schema_rec` and short-circuit on repeat (or a max-depth counter as a cheaper stopgap), plus a regression test with a self-referential schema. A defensive `except RecursionError` at the call sites would also be a reasonable belt-and-suspenders addition either way.
Contributor guide
Assessment
This issue has not been assessed yet.