[CH] simdjson doesn't support strings which contain invalid unicodes
Open
bug
triage
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
CH (ClickHouse)
### Bug description
[Expected behavior] and [actual behavior].
simdjson fails to parse following json
```
{"a":"431924697b1\ufffdSC\u2995\u06a2\r8\u0010\u06e3\udee4\udff0L\u0001Y"}
```
`\udee4` and `\udff0` are invalid unicodes
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
Contributor guide
Research direction
Start by reproducing the reported JSON through the ClickHouse simdjson path, focusing on the invalid surrogate sequences \udee4 and \udff0. Clarify the expected and actual behavior, then verify that the same input is handled correctly without breaking valid JSON parsing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100