Aiven-Open / Aiven-Open/karapace
Harden JSON Schema handling against ReDoS from user-supplied schema regexes
- 主要語言
- Python
- 星號
- 634
- 分支
- 110
- 平均合併
- 4 天 7 小時
- 30 天內合併 PR
- 4
描述
# What happened?
Two code paths compile/run a regular expression that comes from a **user-registered JSON
schema**, with no timeout or complexity bound. A schema containing a catastrophic-backtracking
pattern therefore lets a single request pin a CPU core for a long time (ReDoS). Filing this as
a robustness bug rather than via the security program, since `SECURITY.md` scopes DoS out.
**1. Compatibility check** — `src/karapace/core/compatibility/jsonschema/utils.py:436`
```python
def schema_from_partially_open_content_model(schema, target_property_name):
for pattern, pattern_schema in schema.get(Keyword.PATTERN_PROPERTIES.value, {}).items():
if re.match(pattern, target_property_name): # pattern + string both from user schemas
...
```
`pattern` is a `patternProperties` key of a registered schema and `target_property_name` is a
property name from another version; reached from `compatibility/jsonschema/checks.py:682` and
`:727` during compatibility checking on registration.
**2. Value validation** — `src/karapace/core/serialization.py:522` (`read_value`) and `:554`
(`write_value`)
```python
schema.schema.validate(value) # jsonschema evaluates the schema's 'pattern' over the value
```
The registered schema's `pattern`/`patternProperties` and the message `value` are both
user-supplied.
**Small reproducer** (the underlying primitive both paths hit):
```python
import re, time
p = r"(a+)+$" # a valid patternProperties key / schema "pattern"
for n in (20, 24, 28):
s = "a" * n + "!"
t = time.time(); re.match(p, s); print(n, round(time.time() - t, 3), "s")
# ~0.04s, ~0.8s, >8s -- time roughly doubles per extra character
```
Confirmed against the real `jsonschema==4.26.0` validator for path (2) as well.
# What did you expect to happen?
Regexes taken from untrusted schemas should be evaluated in bounded time rather than being able
to hang the worker. Reasonable options:
- evaluate schema-derived regexes with a linear-time engine (e.g. `google-re2`), or
- apply a match timeout, or
- bound the matched string length and/or reject non-linear patterns at registration time.
# What else do we need to know?
- Reproduced on `main` @ `54a4680d1925d84b6cc221e74f9744a716731a33`, `jsonschema==4.26.0` (as pinned in `requirements/requirements.txt`).
- Both paths run synchronously in their request handlers (no per-request CPU bound); the protobuf
path is already isolated in a 10s-bounded subprocess, these two are not.
- Happy to open a PR for whichever mitigation you'd prefer.
貢獻指南
研究方向
Start with the reproducer, then inspect schema_from_partially_open_content_model in src/karapace/core/compatibility/jsonschema/utils.py and read_value/write_value in src/karapace/core/serialization.py, along with the compatibility callers in checks.py. Trace both user-schema regex paths and establish a mitigation that keeps regex evaluation bounded; done means the reproducer no longer causes unbounded synchronous CPU use in either path.
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- backend, security
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 活躍
- 描述清晰度
- 基本清楚
- 新手友好度
- 48/100