cbor_value_to_json_advance() does not perform the documented UTF-8 validation
- Lingua principale
- C
- Stelle
- 630
- Fork
- 222
- Merge medio
- 1g 3h
- PR unite (30g)
- 1
Descrizione
`cbor_value_to_json_advance()` is documented to reject text strings that are not
valid UTF-8, but it does not check. It returns `CborNoError` and copies the bytes
straight into the JSON output, so the result is not valid JSON.
The documentation in `src/cbortojson.c` says:
> These functions also perform UTF-8 validation in CBOR text strings. If they
> encounter a sequence of bytes that is not permitted in UTF-8, they will return
> `CborErrorInvalidUtf8TextString`. That includes encoding of surrogate points in
> UTF-8.
`cbor_value_to_pretty_advance()` does exactly that, via `utf8EscapedDump()` in
`src/cborpretty.c`, which calls `get_utf8()` and returns
`CborErrorInvalidUtf8TextString` on failure. The JSON path does not: text strings
go through `escape_text_string()` in `src/cbortojson.c`, which walks the bytes
escaping the characters JSON requires escaping and never decodes UTF-8.
### Reproducer
```c
#include
#include "cbor.h"
#include "cborjson.h"
int main(void)
{
/* CBOR text string of one byte, 0xFF, which cannot start a UTF-8 sequence */
static const uint8_t data[] = { 0x61, 0xff };
CborParser parser;
CborValue it;
CborError err;
err = cbor_parser_init(data, sizeof(data), 0, &parser, &it);
if (!err)
err = cbor_value_to_json_advance(stdout, &it, 0);
fprintf(stderr, "\nerr = %d (%s)\n", (int)err, cbor_error_string(err));
return 0;
}
```
Observed:
```
"<0xff>"
err = 0 (no error)
```
Expected, per the documentation quoted above: `CborErrorInvalidUtf8TextString`
(516).
### Other inputs
Same result for every ill-formed sequence I tried, including the surrogate case
the documentation calls out specifically. `to_pretty` and `to_json` are given
identical input here:
| CBOR | `cbor_value_to_pretty_advance` | `cbor_value_to_json_advance` |
|---|---|---|
| `61 ff` | 516 InvalidUtf8TextString | 0, emits `"\xff"` |
| `62 c3 28` (bad continuation byte) | 516 | 0, emits `"\xc3("` |
| `63 ed a0 80` (U+D800 as UTF-8) | 516 | 0, emits the surrogate bytes |
| `a1 61 ff 01` (as a map key) | 516 | 0, emits `{"\xff":1}` |
| `82 61 ff 01` (inside an array) | 516 | 0, emits `["\xff",1]` |
Map keys are affected because they go through the same `escape_text_string()`.
### Why it matters beyond the documentation
RFC 8259 §8.1 requires JSON text to be encoded in UTF-8, so the output of a
successful call is not necessarily JSON. A caller that trusts the return value
and hands the buffer to a JSON parser gets a decode error from a function that
reported success. Feeding roughly 27,000 successful conversions from a fuzzing
corpus to a strict JSON parser, every failure was of this kind.
RFC 8949 §3.1 also makes this ill-formed input rather than merely unusual: major
type 3 is defined as a UTF-8 string, and §5.3.2 leaves the decision to reject to
the decoder. `to_pretty` rejects it, so the library has already taken that
position.
### Where
`escape_text_string()` in `src/cbortojson.c` is the place that would need it. It
already iterates the chunk byte by byte, and `get_utf8()` from `src/utf8_p.h` is
what `cborpretty.c` uses for the same job, so the pieces are there.
I have not sent a pull request because there is a choice to make that is yours
rather than mine: validating in `escape_text_string()` changes
`cbor_value_to_json_advance()` from accepting to rejecting these inputs, which is
a behaviour change for anyone currently relying on the pass-through, even though
the documentation has always promised the rejection. The alternative, correcting
the documentation instead, is also self-consistent.
### Version
`9441b2ca88` (current `main` as of 2026-06-19), built with
`CMAKE_BUILD_TYPE=Release`, gcc 15.3.0, x86-64 Linux. `TINYCBOR_VERSION` 7.0.0.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start in src/cbortojson.c at escape_text_string(), then compare its byte handling with get_utf8() in src/utf8_p.h and the validation path used by src/cborpretty.c. Use the reproducer and listed malformed inputs to confirm the chosen behavior; done means the JSON conversion behavior matches the documented UTF-8 contract or the documentation is deliberately corrected.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- c
- Ambito
- backend
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Tranquilla
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 48/100