Guided generation refused for every chunked-static (iOS) export after the first request
- Dominant language
- Swift
- Stars
- 2.1k
- Forks
- 202
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 55
Description
## What happens
Guided generation is refused for chunked-static exports, which is every iOS
export — but only *after* the first request of a session.
Device: iPhone 15 Pro Max, iOS 27.0. Artifact: `Qwen2.5-Coder-1.5B-Instruct`
exported with `coreai.llm.export ... --platform iOS --experimental
--compute-precision float16`. Twelve structured requests through
`LanguageModelSession.respond(to:generating:)`: the first returned a correct
answer, the remaining eleven failed with
```
The selected model does not support guided generation. Consider trying again
with a different model.
```
An earlier artifact (`Qwen3-1.7B`, the `qwen3_1_7b_6bit` preset) showed the same
thing in a different shape: `GeneratedContent does not contain a property
'isCodingRequest'. Content: {}`.
Plain, unstructured generation works on both artifacts and is coherent.
## Cause
`CoreAILanguageModel.isGuidedGenerationSupported` asks whether the resident
engine conforms to `ConstrainedGenerationCapable`, and returns that answer:
```swift
private var isGuidedGenerationSupported: Bool {
if let isConstrainedCapable = resources.loadedEngineIsConstrainedCapable {
return isConstrainedCapable
}
if let supportsLogits = resources.loadedEngineSupportsLogits {
return supportsLogits
}
return true
}
```
`StaticShapeEngine` does not conform to `ConstrainedGenerationCapable`, and does
report `supportsLogits == true`, so the second check is never reached. The
executor already treats the two as alternatives:
```swift
guard engine.supportsLogits || engine is any ConstrainedGenerationCapable else {
throw LanguageModelError.unsupportedCapability(.init(capability: .guidedGeneration, ...))
}
```
The delayed onset is the third branch: with no engine resident both properties
are nil and the model answers `true`, so the first request proceeds and
succeeds.
## Suggested change
Accept either capability, keeping "assume yes" only while nothing is loaded:
```swift
private var isGuidedGenerationSupported: Bool {
let constrainedCapable = resources.loadedEngineIsConstrainedCapable
let supportsLogits = resources.loadedEngineSupportsLogits
if constrainedCapable == nil && supportsLogits == nil { return true }
return (constrainedCapable ?? false) || (supportsLogits ?? false)
}
```
With that change, the same twelve cases return twelve structured answers on the
same artifact and device. Pull requests appear to be closed to outside
contributors, so this is an issue rather than a patch; the change is running
here at mohdelfatih/coreai-models@66531257de45f3c1ffa7618e2950242b853c062d if it
is useful.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.