input-output-hk / input-output-hk/cuddle

Validator does not distinguish bignums (tags 2/3) from plain ints

Open
#207 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Haskell
Stars
7
Forks
5
PR merge metrics
No merged PRs in 30d

Description

# Summary

The validator decodes input with cborg's decodeTerm, which folds tags 2/3 into TInteger at the token level (cborg maps wire headers 0xc2/0xc3 to TypeInteger and decodeInteger consumes the tag + payload). By the time validateTerm runs, the distinction between "major type 0/1" and "tag 2/3 bignum" is erased — but CDDL's matching rules are representation-sensitive: per RFC 8610 (§3.3, Appendix C/D), `uint = #0`, `nint = #1`, and numeric literals match only major type 0/1 data items, never tag 2/3 items. That's why the prelude needs integer = int / bigint and unsigned = uint / biguint.

This matters for interop: encoders other than cborg may legally emit valid-but-non-preferred forms (RFC 8949 §3.4.3 — bignums with leading zeroes MUST be accepted; values that fit major type 0/1 wrapped in tag 2/3 are valid; indefinite-length payloads are valid outside deterministic encoding).

# Observed behavior

Hard failure on valid CBOR:

- `a = biguint` vs `C2 5F 41 05 FF` (tag 2, indefinite-length payload) → `DecodingFailed "expected integer"`. Valid CBOR per RFC 8949 §3.4.3/§2 (definite vs. indefinite is a serialization variant; only deterministic encoding §4.2.1 forbids it). Known issue — see the TODO at Gen.hs (disableTwiddle for tags 2/3).

Wrong results (sign not checked — validateInteger treats Tag 2 and Tag 3 rules identically):

- `a = biguint` vs `C3 49 010000000000000000` (negative bignum) → accepted
- `a = bignint` vs `C2 49 010000000000000000` (positive bignum) → accepted
- `a = biguint` vs `20` (plain −1) → accepted

Too strict (payload bytes are gone, so byte controls run against mempty — see TODO in validateInteger's validateBigInt):

- `a = #6.2(bstr .size 9)` vs `C2 49 010000000000000000` (exactly 9 payload bytes) → rejected

Too lenient (spec deviation, fails open):
- `a = {1: uint}` vs map key encoded as `C2 41 01` → accepted; Appendix C: a literal "matches only a data item with that specific value", and a tag-2 item is a different data-model value

What already works correctly: bignums ≥ 2^64 accepted for `biguint`/`bigint`/`integer` and rejected for `int`/`uint`; leading-zero bignum payloads accepted by value; non-minimal int widths accepted (correct — CDDL matches at the data-model level); generator output is preferred serialization and round-trips.

# Root cause and proposed fix

The information loss is unrecoverable at the Term level in general: `TInteger i` with `|i| < 2^63` can only come from a bignum, but values in `[2^63, 2^64)` (and the negative mirror) decode to `TInteger` from both major type 0/1 and tags 2/3. So a `TInteger`-only fix in the validator cannot be complete, and can't fix the decode failure or byte controls at all.

Proposed: a cuddle-local variant of cborg's decodeTerm that keeps wire bignums as TTagged 2/3 (TBytes …) instead of folding them. This is straightforward because in cborg's decoder primitives, peekTokenType returns TypeInteger exclusively for wire headers 0xc2/0xc3 (genuine ints come back as TypeUInt/TypeUInt64/TypeNInt/TypeNInt64), and decodeTag happily consumes 0xc2/0xc3 as ordinary tags. So the TypeInteger branch can do decodeTag + payload decode instead of decodeInteger. Verified this preserves indefinite-length payloads too (C2 5F 41 05 FF → TTagged 2 (TBytesI …)).

This fixes all four gaps: representation-correct matching for uint/int/literals, sign correctness for biguint/bignint, working `.size`/`.cbor` controls on `#6.2(bstr …)` payloads, and no decode failure on indefinite-length payloads.

Note this establishes a convention on Term that differs from cborg's: TInt/TInteger would mean "major type 0/1 on the wire" and TTagged 2/3 (TBytes …) "bignum on the wire" (cborg's encodeTerm already round-trips this faithfully). The convention must then hold for all Term producers, not just the decoder:

- the generator currently emits TInteger for VBignum literals — it should emit TTagged 2/3 (TBytes minimalBytes);
- the custom-validator API wraps every integer as SingleTerm (TInteger i) and accepts user-constructed terms via validateFromName/validateAgainst — the convention needs documenting at that API boundary (or, more heavyweight, a provenance newtype / dedicated term type, in the spirit of CanonicalTerm).

Value-level equivalence (prelude integer/unsigned, toCanonical map-key dedup, matching literals > 2^64−1 against bignums by numeric value) then becomes an explicit conversion where intended. On that last point: RFC 8610 never actually defines what a literal ≥ 2^64 matches (the basic data model has no such integer) — matching it against tag 2/3 by value is a pragmatic extension, same as the reference cddl tool, and worth stating in the docs.

# Spec references

- RFC 8949 §3.4.3 (bignums: content must be a byte string; leading zeroes MUST be accepted; preferred serialization), §4.2.1/§4.2.2 (deterministic encoding), §5.3.2 (tag validity)
- RFC 8610 §2.2.3 (representation types are data-model level), §3.3 (`uint`/`nint`), Appendix C (`type2 = value`: "matches only a data item with that specific value (no conversions defined)"), Appendix D (prelude: `uint = #0`, `biguint = #6.2(bstr)`, `unsigned = uint / biguint`, …)

Contributor guide

No contributing guide indexed for this repository

Research direction

Read Gen.hs, validateInteger's validateBigInt, and the cborg decoder primitives first; then trace the custom-validator API and generator handling of VBignum literals. Check how Term producers and consumers represent wire bignums, including indefinite payloads and byte controls. Done means representation-sensitive validation works without losing sign or payload information, and the stated generator and API conventions are addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
compilers, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.