lablup / lablup/backend.ai

Unified Service Discovery with DB-backed Service Catalog

Open
#8,665 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Goal

Replace the current fragmented service discovery mechanisms (Redis hash, etcd, event-based heartbeat) with a unified, DB-backed service catalog that provides:

1. **Consistent registration** — All components (Agent, Manager, Storage Proxy, AppProxy, Webserver) register via Redis events
1. **Queryable catalog** — DB-backed `service_catalog` and `service_catalog_endpoint` tables, exposed via GraphQL for user-facing service status views
1. **Multi-endpoint support** — Each service registers multiple endpoints with `role` (api, rpc, metrics, etc.) and `scope` (host, container, cluster, public) dimensions
1. **Config-driven endpoints** — Fully explicit `[service-discovery.endpoints]` TOML config section with CLI migration tool for existing deployments
1. **Resilient client pools** — SD-based client pools with in-memory cache and self-preservation mode (Eureka/Consul patterns)

## Current State

|Component|Registration Method|Backend|
|---|---|---|
|Agent|Heartbeat event|Event → DB|
|Manager|ServiceDiscoveryLoop|ETCD / Redis|
|Storage Proxy|ServiceDiscoveryLoop|ETCD / Redis|
|AppProxy|ServiceDiscoveryLoop|Redis only|
|Webserver|ServiceDiscoveryLoop|ETCD / Redis|

## Target State

All components → `ServiceRegisteredEvent` (Redis Stream) → Manager consumes → DB catalog → GraphQL query

## Phases

- Phase 1: Common foundation (config models, event types, DB models)
- Phase 2: CLI migration tool (`config migrate-sd`)
- Phase 3: Runtime integration (event publishing, Manager handler, GraphQL API)
- Phase 4: SD-based client pool (ServiceCatalogCache + SDClientPool)

## Key Design Decisions

- **Redis events as transport**: Leverages existing event infrastructure (Redis Stream DB 4), no new config needed for service-to-service communication
- **DB as authoritative store**: Enables SQL queries, GraphQL exposure, audit trails
- **role + scope endpoint model**: Supports diverse network topologies (host, container, k8s cluster, public LB)
- **Config-driven (Option B)**: Explicit `[service-discovery.endpoints]` section; CLI migration tool generates from existing config fields
- **Self-preservation on backend failure**: Freeze last known state instead of deregistering services (inspired by Netflix Eureka)

JIRA Issue: BA-4313

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.