lablup / lablup/backend.ai

Migrate Agent/AgentNode live_stat GQL resolvers from Valkey to Prometheus

Open
#11,981 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

BA-5744 migrated the container/kernel live_stat GQL resolvers to Prometheus via MetricService.query_container_live_stats -> MetricRepository.query_container_live_stats. The agent-level resolvers were left on Valkey and must be migrated to complete the BA-4039 effort.

Current state (src/ai/backend/manager/api/gql_legacy/agent.py):
- Agent (class at line 338) resolve_live_stat -> batch_load_live_stat -> ctx.valkey_stat.get_agent_statistics_batch
- AgentNode (class at line 126) uses the same Valkey path

Scope:
- Add a node-level Prometheus query that aggregates container metrics per agent (by agent/instance label), batched for N agents in a single query. Reuse the established Action/Service/Repository pattern from BA-5744.
- Rewire both Agent and AgentNode live_stat resolvers to the new Prometheus-backed path.
- Remove the ctx.valkey_stat.get_agent_statistics_batch call from the resolvers.

Note: the existing Prometheus path is container-level (kernel_ids). Agent live_stat is node-level aggregation across an agent's containers, so a new agent/node-level query is required rather than a direct copy of the container query.

## Success Criteria
- [ ] Agent.live_stat returns Prometheus-sourced data with the same JSON shape as before
- [ ] AgentNode.live_stat returns Prometheus-sourced data via the same path
- [ ] New repository method aggregates metrics per agent (node-level), batching N agents in a single query
- [ ] ctx.valkey_stat.get_agent_statistics_batch is no longer called from the resolvers
- [ ] pants test passes for affected manager packages

JIRA Issue: BA-6325

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.