lablup / lablup/backend.ai

Implement SD-based client pool with cache and self-preservation

Open
#8,670 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Objective

Implement a service catalog cache and SD-based client pool that provides resilient inter-service communication with graceful degradation.

## Scope

### 1. ServiceCatalogCache

- In-memory cache of `service_catalog` data
- Full fetch on startup from Manager API or DB (for Manager itself)
- Periodic delta sync (every 30-60s)
- **Self-preservation mode** (Eureka pattern): on backend failure, freeze last known state instead of evicting services
- Staleness indicator on cached data
- Optional disk snapshot for restart recovery

### 2. SDClientPool

- Built on top of `ServiceCatalogCache`
- Select endpoints by `(service_group, role, scope)`
- Manage HTTP (`aiohttp.ClientSession`) and RPC (`PeerInvoker`) connections
- Auto-sync with cache changes: add new services, remove deregistered ones
- Load balancing: round-robin or least-connections within a service group
- Existing connections survive cache sync (data plane / control plane separation)

### 3. Integration Points

- Replace hardcoded storage proxy addresses in Manager with `SDClientPool("storage-proxy", role="manager-api", scope="cluster")`
- Replace `AgentClientPool` address resolution with SD-based lookup (keep ZMQ connection logic)
- Replace AppProxy client address lookup with SD-based resolution

## Acceptance Criteria

- [ ] `ServiceCatalogCache` caches and syncs service catalog
- [ ] Self-preservation mode activates on backend failure, freezes state
- [ ] `SDClientPool` selects endpoints by role/scope
- [ ] Existing connections maintained during cache sync
- [ ] Tests for cache sync, self-preservation, and pool lifecycle

JIRA Issue: BA-4318

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.