lablup / lablup/backend.ai

Avoid N+1 circuit queries when fetching App Proxy worker information

Open
#5,301 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

The current Manager → App Proxy API includes the worker listing & detail queries that runs an N+1 worker-circuit queries, which may increase tail latency of the Manager and App Proxy APIs with a large number of circuits.

**We need to avoid this kind of queries and update Control Panel and Manager NOT TO RELY ON individual circuit information fetched like this.**

Instead, we have to:

- Offload or optimize the search for vacant circuits. (e.g., indexing based on Radix Tree?)
- Shrink the scope of circuit list queries to individual endpoints.
- Split worker information queries and circuit list queries. Use the latter sparingly as much as possible.

To keep App Proxy API compatibility, we could just let the `coordinator.models.worker.Worker.list_slots()` method to return an empty list. Afterwards, we MUST REDESIGN the App Proxy coordinator API.

JIRA Issue: BA-1985

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.