Aiven-Open / Aiven-Open/karapace

Harden JSON Schema handling against ReDoS from user-supplied schema regexes

未關閉
#1,362 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
634
分支
110
平均合併
4 天 7 小時
30 天內合併 PR
4

描述

# What happened?

Two code paths compile/run a regular expression that comes from a **user-registered JSON
schema**, with no timeout or complexity bound. A schema containing a catastrophic-backtracking
pattern therefore lets a single request pin a CPU core for a long time (ReDoS). Filing this as
a robustness bug rather than via the security program, since `SECURITY.md` scopes DoS out.

**1. Compatibility check** — `src/karapace/core/compatibility/jsonschema/utils.py:436`
```python
def schema_from_partially_open_content_model(schema, target_property_name):
for pattern, pattern_schema in schema.get(Keyword.PATTERN_PROPERTIES.value, {}).items():
if re.match(pattern, target_property_name): # pattern + string both from user schemas
...
```
`pattern` is a `patternProperties` key of a registered schema and `target_property_name` is a
property name from another version; reached from `compatibility/jsonschema/checks.py:682` and
`:727` during compatibility checking on registration.

**2. Value validation** — `src/karapace/core/serialization.py:522` (`read_value`) and `:554`
(`write_value`)
```python
schema.schema.validate(value) # jsonschema evaluates the schema's 'pattern' over the value
```
The registered schema's `pattern`/`patternProperties` and the message `value` are both
user-supplied.

**Small reproducer** (the underlying primitive both paths hit):
```python
import re, time
p = r"(a+)+$" # a valid patternProperties key / schema "pattern"
for n in (20, 24, 28):
s = "a" * n + "!"
t = time.time(); re.match(p, s); print(n, round(time.time() - t, 3), "s")
# ~0.04s, ~0.8s, >8s -- time roughly doubles per extra character
```
Confirmed against the real `jsonschema==4.26.0` validator for path (2) as well.

# What did you expect to happen?

Regexes taken from untrusted schemas should be evaluated in bounded time rather than being able
to hang the worker. Reasonable options:
- evaluate schema-derived regexes with a linear-time engine (e.g. `google-re2`), or
- apply a match timeout, or
- bound the matched string length and/or reject non-linear patterns at registration time.

# What else do we need to know?

- Reproduced on `main` @ `54a4680d1925d84b6cc221e74f9744a716731a33`, `jsonschema==4.26.0` (as pinned in `requirements/requirements.txt`).
- Both paths run synchronously in their request handlers (no per-request CPU bound); the protobuf
path is already isolated in a 10s-bounded subprocess, these two are not.
- Happy to open a PR for whichever mitigation you'd prefer.

貢獻指南

開啟貢獻指南

研究方向

Start with the reproducer, then inspect schema_from_partially_open_content_model in src/karapace/core/compatibility/jsonschema/utils.py and read_value/write_value in src/karapace/core/serialization.py, along with the compatibility callers in checks.py. Trace both user-schema regex paths and establish a mitigation that keeps regex evaluation bounded; done means the reproducer no longer causes unbounded synchronous CPU use in either path.

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
backend, security
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
活躍
描述清晰度
基本清楚
新手友好度
48/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。