lablup / lablup/backend.ai

Remove PostgreSQL dependency from App Proxy Coordinator

Open
#7,065 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Goal

Remove PostgreSQL database dependency from App Proxy Coordinator and replace it with Redis-only state management.

## Architecture Change

**Before:**

```
Manager ──(REST API)──> App Proxy Coordinator ──(PostgreSQL)──> State Storage
```

**After:**

```
Manager ──(Redis push)──> Redis <──(read)── App Proxy Coordinator
```

## Key Changes

1. **Data Flow**: Manager pushes route info directly to Redis, Coordinator reads from Redis
1. **State Management**: All state stored in Redis with TTL-based cleanup
1. **Recovery**: Workers re-register on coordinator restart, Manager repushes routes

## Database Tables to Remove

|Table|Replacement|
|---|---|
|`workers`|Redis hash + TTL heartbeat|
|`circuits`|Manager pushes to Redis|
|`endpoints`|Manager pushes to Redis|
|`tokens`|Redis hash + TTL|
|`worker_app_filters`|Stored with worker in Redis|

## Benefits

- Simplified deployment (no PostgreSQL required)
- Reduced operational complexity
- Faster state access (in-memory)
- Better alignment with ephemeral nature of routing state

JIRA Issue: BA-3221

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.