input-output-hk / input-output-hk/cuddle
Validator does not distinguish bignums (tags 2/3) from plain ints
- Dominant language
- Haskell
- Stars
- 7
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
# Summary
The validator decodes input with cborg's decodeTerm, which folds tags 2/3 into TInteger at the token level (cborg maps wire headers 0xc2/0xc3 to TypeInteger and decodeInteger consumes the tag + payload). By the time validateTerm runs, the distinction between "major type 0/1" and "tag 2/3 bignum" is erased — but CDDL's matching rules are representation-sensitive: per RFC 8610 (§3.3, Appendix C/D), `uint = #0`, `nint = #1`, and numeric literals match only major type 0/1 data items, never tag 2/3 items. That's why the prelude needs integer = int / bigint and unsigned = uint / biguint.
This matters for interop: encoders other than cborg may legally emit valid-but-non-preferred forms (RFC 8949 §3.4.3 — bignums with leading zeroes MUST be accepted; values that fit major type 0/1 wrapped in tag 2/3 are valid; indefinite-length payloads are valid outside deterministic encoding).
# Observed behavior
Hard failure on valid CBOR:
- `a = biguint` vs `C2 5F 41 05 FF` (tag 2, indefinite-length payload) → `DecodingFailed "expected integer"`. Valid CBOR per RFC 8949 §3.4.3/§2 (definite vs. indefinite is a serialization variant; only deterministic encoding §4.2.1 forbids it). Known issue — see the TODO at Gen.hs (disableTwiddle for tags 2/3).
Wrong results (sign not checked — validateInteger treats Tag 2 and Tag 3 rules identically):
- `a = biguint` vs `C3 49 010000000000000000` (negative bignum) → accepted
- `a = bignint` vs `C2 49 010000000000000000` (positive bignum) → accepted
- `a = biguint` vs `20` (plain −1) → accepted
Too strict (payload bytes are gone, so byte controls run against mempty — see TODO in validateInteger's validateBigInt):
- `a = #6.2(bstr .size 9)` vs `C2 49 010000000000000000` (exactly 9 payload bytes) → rejected
Too lenient (spec deviation, fails open):
- `a = {1: uint}` vs map key encoded as `C2 41 01` → accepted; Appendix C: a literal "matches only a data item with that specific value", and a tag-2 item is a different data-model value
What already works correctly: bignums ≥ 2^64 accepted for `biguint`/`bigint`/`integer` and rejected for `int`/`uint`; leading-zero bignum payloads accepted by value; non-minimal int widths accepted (correct — CDDL matches at the data-model level); generator output is preferred serialization and round-trips.
# Root cause and proposed fix
The information loss is unrecoverable at the Term level in general: `TInteger i` with `|i| < 2^63` can only come from a bignum, but values in `[2^63, 2^64)` (and the negative mirror) decode to `TInteger` from both major type 0/1 and tags 2/3. So a `TInteger`-only fix in the validator cannot be complete, and can't fix the decode failure or byte controls at all.
Proposed: a cuddle-local variant of cborg's decodeTerm that keeps wire bignums as TTagged 2/3 (TBytes …) instead of folding them. This is straightforward because in cborg's decoder primitives, peekTokenType returns TypeInteger exclusively for wire headers 0xc2/0xc3 (genuine ints come back as TypeUInt/TypeUInt64/TypeNInt/TypeNInt64), and decodeTag happily consumes 0xc2/0xc3 as ordinary tags. So the TypeInteger branch can do decodeTag + payload decode instead of decodeInteger. Verified this preserves indefinite-length payloads too (C2 5F 41 05 FF → TTagged 2 (TBytesI …)).
This fixes all four gaps: representation-correct matching for uint/int/literals, sign correctness for biguint/bignint, working `.size`/`.cbor` controls on `#6.2(bstr …)` payloads, and no decode failure on indefinite-length payloads.
Note this establishes a convention on Term that differs from cborg's: TInt/TInteger would mean "major type 0/1 on the wire" and TTagged 2/3 (TBytes …) "bignum on the wire" (cborg's encodeTerm already round-trips this faithfully). The convention must then hold for all Term producers, not just the decoder:
- the generator currently emits TInteger for VBignum literals — it should emit TTagged 2/3 (TBytes minimalBytes);
- the custom-validator API wraps every integer as SingleTerm (TInteger i) and accepts user-constructed terms via validateFromName/validateAgainst — the convention needs documenting at that API boundary (or, more heavyweight, a provenance newtype / dedicated term type, in the spirit of CanonicalTerm).
Value-level equivalence (prelude integer/unsigned, toCanonical map-key dedup, matching literals > 2^64−1 against bignums by numeric value) then becomes an explicit conversion where intended. On that last point: RFC 8610 never actually defines what a literal ≥ 2^64 matches (the basic data model has no such integer) — matching it against tag 2/3 by value is a pragmatic extension, same as the reference cddl tool, and worth stating in the docs.
# Spec references
- RFC 8949 §3.4.3 (bignums: content must be a byte string; leading zeroes MUST be accepted; preferred serialization), §4.2.1/§4.2.2 (deterministic encoding), §5.3.2 (tag validity)
- RFC 8610 §2.2.3 (representation types are data-model level), §3.3 (`uint`/`nint`), Appendix C (`type2 = value`: "matches only a data item with that specific value (no conversions defined)"), Appendix D (prelude: `uint = #0`, `biguint = #6.2(bstr)`, `unsigned = uint / biguint`, …)
Contributor guide
No contributing guide indexed for this repository
Research direction
Read Gen.hs, validateInteger's validateBigInt, and the cborg decoder primitives first; then trace the custom-validator API and generator handling of VBignum literals. Check how Term producers and consumers represent wire bignums, including indefinite payloads and byte controls. Done means representation-sensitive validation works without losing sign or payload information, and the stated generator and API conventions are addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- compilers, testing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100