lablup / lablup/backend.ai

Remove Agent-side model definition generation; use Manager-provided definition only

Open
#10,983 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Background

After BA-5665, Agent now accepts model_definition from Manager via internal_data when available, but still keeps variant-specific hardcoded generation (vllm, tgi, nim, sglang, modular-max, cmd) and VFolder reading (custom variant) as fallback paths. This creates dual responsibility for model definition generation between Manager and Agent.

## Objective

Make Manager the single source of truth for model definition generation. Agent should only consume the definition provided via internal_data, with no local generation or VFolder reading.

## Scope

- Agent agent.py: Remove match runtime_variant block (variant-specific hardcoded definitions) and custom VFolder reading from load_model_definition(). Use only internal_data["model_definition"].
- Agent model_service.py (stage): Remove variant-specific _get_model_definition_from_\* methods. Use only spec.model_definition.
- Manager (legacy path): Ensure POST /services/ and event_dispatcher always generate and pass model_definition, even for legacy endpoints where deployment_revisions.model_definition is NULL.
- Remove MODEL_SERVICE_RUNTIME_PROFILES dependency from Agent if no longer needed.

## Acceptance Criteria

- Agent has no model definition generation or VFolder reading logic
- All model service creation paths (sokovan + legacy) provide model_definition via internal_data
- Existing model services continue to work (backward compatibility for legacy endpoints)

JIRA Issue: BA-5680

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.