Layr-Labs / Layr-Labs/d-inference
[bug] mlx-community/gemma-4-26b-a4b-it-8bit chat_template is not portable to swift-jinja
- Dominant language
- Go
- Stars
- 522
- Forks
- 113
- Avg merge
- 17h 26m
- Merged PRs (30d)
- 111
Description
### Component
provider-swift (Swift CLI)
### What happened?
When tool definitions are included in calls to /v1/chat/completions to providers using the swift-jinja template interpreter, `500 upper filter requires string` is always thrown. This impacts all agent harnesses that send tools. This occurs because the chat_template.jinja for this particular model is defective and only succeeds for other services due to undocumented permissiveness in the CPython jinja2 used by oMLX and other backends.
Defect specifics:
- CPython jinja2 is permissive about `Undefined | upper` and propagates `Undefined / ""` through the filter without raising
Patch description:
- throughout chat_template.jinja, replace `X | upper` with `(X | default('')) | upper` to properly treat `missing/None` as the empty string, failing all comparisons without crashing
Darkbloom actions:
- must patch chat_template.jinja
- must re-vend the patched chat_template.jinja from a new R2 prefix
- must bump `aggregate_sha256` in the coordinator catalog
- should consider upstreaming the patch to mlx-community/gemma-4-26b-a4b-it-8bit
- should expand test coverage to include cases that send tool definitions
- should include defensive try/catch in provider-swift's `MultiModelBatchSchedulerEngine` to downgrade the cascading `model load failed` into a clean 4xx error
### Expected behavior
Chat completions for mlx-community/gemma-4-26b-a4b-it-8bit from harnesses that send tool definitions should succeed for Darkbloom as they succeed when this model is loaded locally in oMLX
### Steps to reproduce
set Darkbloom as a provider in any harness that sends tool definitions
### Environment
computers
### Logs
```shell
```
### Confirmations
- [x] I searched existing issues and this isn't a duplicate
Contributor guide
Research direction
Start with the model's chat_template.jinja and provider-swift's MultiModelBatchSchedulerEngine, then inspect the coordinator catalog and existing tests. Reproduce a chat completion that includes tool definitions, verify the patched template is re-vended from a new R2 prefix and the catalog aggregate_sha256 is updated, and add coverage for tool calls; a clean 4xx for model-load failures is also suggested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- ai, api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100