intel / intel/tinycbor

cbor_value_to_json_advance() does not perform the documented UTF-8 validation

Open
#331 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
630
Forks
222
Avg merge
1d 3h
Merged PRs (30d)
1

Description

`cbor_value_to_json_advance()` is documented to reject text strings that are not
valid UTF-8, but it does not check. It returns `CborNoError` and copies the bytes
straight into the JSON output, so the result is not valid JSON.

The documentation in `src/cbortojson.c` says:

> These functions also perform UTF-8 validation in CBOR text strings. If they
> encounter a sequence of bytes that is not permitted in UTF-8, they will return
> `CborErrorInvalidUtf8TextString`. That includes encoding of surrogate points in
> UTF-8.

`cbor_value_to_pretty_advance()` does exactly that, via `utf8EscapedDump()` in
`src/cborpretty.c`, which calls `get_utf8()` and returns
`CborErrorInvalidUtf8TextString` on failure. The JSON path does not: text strings
go through `escape_text_string()` in `src/cbortojson.c`, which walks the bytes
escaping the characters JSON requires escaping and never decodes UTF-8.

### Reproducer

```c
#include
#include "cbor.h"
#include "cborjson.h"

int main(void)
{
/* CBOR text string of one byte, 0xFF, which cannot start a UTF-8 sequence */
static const uint8_t data[] = { 0x61, 0xff };
CborParser parser;
CborValue it;
CborError err;

err = cbor_parser_init(data, sizeof(data), 0, &parser, &it);
if (!err)
err = cbor_value_to_json_advance(stdout, &it, 0);
fprintf(stderr, "\nerr = %d (%s)\n", (int)err, cbor_error_string(err));
return 0;
}
```

Observed:

```
"<0xff>"
err = 0 (no error)
```

Expected, per the documentation quoted above: `CborErrorInvalidUtf8TextString`
(516).

### Other inputs

Same result for every ill-formed sequence I tried, including the surrogate case
the documentation calls out specifically. `to_pretty` and `to_json` are given
identical input here:

| CBOR | `cbor_value_to_pretty_advance` | `cbor_value_to_json_advance` |
|---|---|---|
| `61 ff` | 516 InvalidUtf8TextString | 0, emits `"\xff"` |
| `62 c3 28` (bad continuation byte) | 516 | 0, emits `"\xc3("` |
| `63 ed a0 80` (U+D800 as UTF-8) | 516 | 0, emits the surrogate bytes |
| `a1 61 ff 01` (as a map key) | 516 | 0, emits `{"\xff":1}` |
| `82 61 ff 01` (inside an array) | 516 | 0, emits `["\xff",1]` |

Map keys are affected because they go through the same `escape_text_string()`.

### Why it matters beyond the documentation

RFC 8259 §8.1 requires JSON text to be encoded in UTF-8, so the output of a
successful call is not necessarily JSON. A caller that trusts the return value
and hands the buffer to a JSON parser gets a decode error from a function that
reported success. Feeding roughly 27,000 successful conversions from a fuzzing
corpus to a strict JSON parser, every failure was of this kind.

RFC 8949 §3.1 also makes this ill-formed input rather than merely unusual: major
type 3 is defined as a UTF-8 string, and §5.3.2 leaves the decision to reject to
the decoder. `to_pretty` rejects it, so the library has already taken that
position.

### Where

`escape_text_string()` in `src/cbortojson.c` is the place that would need it. It
already iterates the chunk byte by byte, and `get_utf8()` from `src/utf8_p.h` is
what `cborpretty.c` uses for the same job, so the pieces are there.

I have not sent a pull request because there is a choice to make that is yours
rather than mine: validating in `escape_text_string()` changes
`cbor_value_to_json_advance()` from accepting to rejecting these inputs, which is
a behaviour change for anyone currently relying on the pass-through, even though
the documentation has always promised the rejection. The alternative, correcting
the documentation instead, is also self-consistent.

### Version

`9441b2ca88` (current `main` as of 2026-06-19), built with
`CMAKE_BUILD_TYPE=Release`, gcc 15.3.0, x86-64 Linux. `TINYCBOR_VERSION` 7.0.0.

Contributor guide

Open the contributing guide

Research direction

Start in src/cbortojson.c at escape_text_string(), then compare its byte handling with get_utf8() in src/utf8_p.h and the validation path used by src/cborpretty.c. Use the reproducer and listed malformed inputs to confirm the chosen behavior; done means the JSON conversion behavior matches the documented UTF-8 contract or the documentation is deliberately corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.