`cmd` runtime variant ignores model-definition.yaml, uses empty start_command
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
When using the `cmd` runtime variant for a deployment, the agent ignores the `model-definition.yaml` file in the model vfolder and generates a hardcoded model definition with an empty `start_command` and port 8000.
This results in the model service never starting inside the container, causing readiness checks to fail permanently.
Code location: `src/ai/backend/agent/agent.py:3395-3404`
```python
case "cmd":
_model = {
"name": "image-model",
"model_path": model_folder.kernel_path.as_posix(),
"service": {
"start_command": image_command, # empty for generic python images
"port": 8000,
},
}
raw_definition = {"models": [_model]}
```
The `cmd` variant uses `image_command` (extracted from the image label `ai.backend.service-ports`), which is empty for standard Python images. Meanwhile, the `model-definition.yaml` (which contains the correct `start_command` and `port`) is only read by the `custom` variant.
Additionally, the `model_definition` field explicitly set in the deployment revision JSON (via `--initial-revision`) is stored in the DB but completely ignored by the agent — the agent always regenerates the definition based on the runtime variant.
This behavior is unintuitive: users expect either the YAML file or the explicitly provided model_definition to take effect, but neither does with `cmd`.
Workaround: Use the `custom` runtime variant instead, which reads `model-definition.yaml` from the vfolder.
JIRA Issue: BA-5644
Contributor guide
Assessment
This issue has not been assessed yet.