anthropics / anthropics/ClaudeForFoundationModels
RequestBuilder drops `x-order` and randomizes `properties` order, silently changing guided-generation results
- Lenguaje dominante
- Swift
- Estrellas
- 292
- Forks
- 30
- Merge medio
- 23 min
- PR fusionados (30 d)
- 4
Descripción
### Summary
`RequestBuilder.jsonSchema(from:)` sends a `GenerationSchema`'s `properties` in a **non-deterministic order that changes on every process launch**. Under `output_config.format` the model emits properties in schema order, so this is not cosmetic: an optional field's position materially decides whether it gets populated.
In our app an optional `Double?` came back `nil` on essentially every request for three weeks. The field, the guide text, the image and the model were all fine — it was landing in the first one or two properties.
### Cause
Two independent order-losing steps:
1. `GenerationSchema`'s `Codable` conformance encodes `properties` as an unordered dictionary and preserves declaration order in a separate **`x-order`** key. `allowedSchemaKeys` does not include `x-order`, so `sanitize` drops it.
2. `JSONValue.encoded(_:)` round-trips through `JSONDecoder` into `JSONValue.object([String: JSONValue])` — a Swift `Dictionary`. `sanitize` then builds another one. Swift seeds `Hashable` per process, so the surviving order is redrawn each launch and fixed for that launch's lifetime.
### Reproduction
Encode any multi-field `@Generable` type's `generationSchema`, run it through the same allowlist + `additionalProperties: false` steps, and print the key order from the encoded bytes. Four consecutive launches of one unchanged build, a six-field type:
```
run 1 fillLevel, volumeML, abv, brand, name, category
run 2 brand, abv, fillLevel, name, category, volumeML
run 3 abv, volumeML, fillLevel, category, brand, name
run 4 name, category, abv, brand, fillLevel, volumeML
x-order brand, category, name, abv, volumeML, fillLevel ← declared order, dropped
```
### Impact, measured
Same image, same system prompt, same `@Guide` strings, `claude-opus-5`, via `output_config.format`, sweeping one optional `Double?` (`abv`, a percentage read off a bottle label) through all six positions — 3 runs each:
| position | correct value returned |
|---|---|
| 1 of 6 | **0/3** |
| 2 of 6 | **0/3** |
| 3 of 6 | 3/3 |
| 4 of 6 | 3/3 |
| 5 of 6 | 3/3 |
| 6 of 6 | 3/3 |
With the declared (`x-order`) order restored, the full six-field schema returns the correct value **5/5**. A three-field schema shows the same shape: first 0/5, middle 5/5, last 5/5.
The field's guide instructs the model to omit rather than guess when it cannot read the digits. In the leading position it has to commit to the number before generating anything else about the subject, so it omits — reasonable behaviour given an unreasonable schema order.
Ruled out along the way, each against raw `api.anthropic.com` with identical prompt and guide text: the API surface (tool use and `output_config.format` both fine), image downscaling (fine at full resolution and at `ClaudeImage`'s 1.15 MP budget), guide wording, and the field being absent or non-emittable.
### Suggested fix
Preserve declaration order through to the wire. Either add `x-order` handling to `jsonSchema(from:)` — reorder `properties` by it, then drop the key — or carry `properties` in an order-preserving representation rather than `[String: JSONValue]`. The API doesn't accept `x-order` itself, so it does need removing; only the ordering it describes has to survive.
A regression test asserting that the emitted `properties` order equals the type's declaration order across repeated encodes would catch this — the per-process hash seed means a single-run test can pass by luck.
### Environment
- `ClaudeForFoundationModels` @ `bd4591338e0c3f4798ea38be57ce77ab6934ffb3` (PR #24, 2026-08-13)
- Xcode 27 beta / iOS 27 SDK, `claude-opus-5` and `claude-haiku-4-5`
### Aside, not a bug
`sanitize` also strips `minimum`/`maximum`, which is correct — the API rejects them (`"For 'number' type, properties maximum, minimum are not supported"`). Worth a note in the docs though: it means `@Guide(..., .range(...))` is silently inert on this path, and callers have to enforce ranges themselves. We were getting absurd integers (a 99-digit volume in one probe) before realising the constraint never reached the model.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.