lablup / lablup/backend.ai

`cmd` runtime variant ignores model-definition.yaml, uses empty start_command

Open
#10,895 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

When using the `cmd` runtime variant for a deployment, the agent ignores the `model-definition.yaml` file in the model vfolder and generates a hardcoded model definition with an empty `start_command` and port 8000.

This results in the model service never starting inside the container, causing readiness checks to fail permanently.

Code location: `src/ai/backend/agent/agent.py:3395-3404`

```python

case "cmd":

_model = {

"name": "image-model",

"model_path": model_folder.kernel_path.as_posix(),

"service": {

"start_command": image_command, # empty for generic python images

"port": 8000,

},

}

raw_definition = {"models": [_model]}

```

The `cmd` variant uses `image_command` (extracted from the image label `ai.backend.service-ports`), which is empty for standard Python images. Meanwhile, the `model-definition.yaml` (which contains the correct `start_command` and `port`) is only read by the `custom` variant.

Additionally, the `model_definition` field explicitly set in the deployment revision JSON (via `--initial-revision`) is stored in the DB but completely ignored by the agent — the agent always regenerates the definition based on the runtime variant.

This behavior is unintuitive: users expect either the YAML file or the explicitly provided model_definition to take effect, but neither does with `cmd`.

Workaround: Use the `custom` runtime variant instead, which reads `model-definition.yaml` from the vfolder.

JIRA Issue: BA-5644

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.