ElementsProject / ElementsProject/lightning

msggen: Add UnionField support to generators for oneOf schemas

Open
#8,964 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
3.1k
Forks
1k
Avg merge
4d 10h
Merged PRs (30d)
13

Description

## Problem

`msggen` has a `UnionField` class (`model.py:369-400`) designed to represent `oneOf` JSON schemas, but no generator in the pipeline can actually render it. As a result, every schema field that uses `oneOf` is manually overridden to a single type (usually `ArrayField` or `PrimitiveField`), silently dropping variants.

This means the typed APIs (cln-rpc, gRPC, Python) cannot express what the C implementation and JSON schema both support. For example, `invoice.exposeprivatechannels` accepts `true`, `["1x1x3"]`, or `"1x1x3"` in JSON-RPC, but cln-rpc only exposes `Option>`.

Related: [ElementsProject/lightning#8961](https://github.com/ElementsProject/lightning/issues/8961)

## Root Cause

1. **`UnionField.from_js()` has a constructor bug** — does not pass `added`/`deprecated` to `__init__`
2. **`CompositeField.from_js()` has no `oneOf` detection** — top-level `oneOf` fields hit the `"type" not in ftype"` warning and are skipped
3. **No generator handles `UnionField`** — all 6 generators (`rust.py`, `proto.py`, `convert.py`, `unconvert.py`, `grpc2py.py`, `notification.py`) only dispatch on `PrimitiveField`, `ArrayField`, `EnumField`, and `CompositeField`
4. **Traversal utilities skip `UnionField`** — `patch.py`, `checks.py`, and both `gather_subfields` functions don't recurse into union variants

Because of (2)-(4), every `oneOf` field requires a manual override in `model.py:537-560`, and every override is lossy.

## Affected Fields (18 schemas use `oneOf`)

| Override | Schema oneOf | What's Lost |
|----------|-------------|-------------|
| `Invoice.exposeprivatechannels` | `boolean \| array[scid] \| scid` | boolean and single scid variants |
| `Invoice.label` (×5 methods) | `string \| integer` | integer variant |
| `Datastore.key` (×4 methods) | `array[string] \| string` | single string variant |
| `Offer.amount` | `msat_or_any \| currency` | currency variant |
| `SetConfig.val` | `string \| integer \| boolean` | integer and boolean variants |
| `Pay.exclude` | `array[scid_dir \| pubkey]` | pubkey variant (array-item oneOf) |

## Fix

1. **`model.py`**: Fix `UnionField.from_js()` constructor, add `oneOf` detection in `CompositeField.from_js()`
2. **`patch.py`, `checks.py`**: Add `UnionField` traversal
3. **`rust.py`**: Generate `#[serde(untagged)]` enums
4. **`proto.py`**: Generate `oneof` blocks with wrapper messages for array variants
5. **`convert.py` / `unconvert.py`**: Generate bidirectional `From` impls for union types
6. **`grpc2py.py`**: Generate `WhichOneof()`-based conversion
7. **Remove 12 overrides** that exist solely because `oneOf` was not handled

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.