lablup / lablup/backend.ai

Implement memory queue and batch writer foundation

Open
#8,541 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Objective

Implement the core infrastructure for batched logging: memory queue management and batch writer.

## Implementation Details

### Components to Create

1. `LogQueueManager` (`src/ai/backend/manager/logging/queue_manager.py`)
- Manages asyncio.Queue instances for each log type
- Provides non-blocking put operations
- Handles queue capacity monitoring
1. `BatchWriter` (`src/ai/backend/manager/logging/batch_writer.py`)
- Background task consuming from queues
- Periodic flush (configurable interval)
- Threshold-based flush (configurable size)
- Uses BulkCreator pattern for DB writes
1. `LogBuffer` (`src/ai/backend/manager/logging/buffer.py`)
- In-memory accumulator for log entries
- Thread-safe operations
- Capacity management

### Configuration

```python
@dataclass
class BatchLoggingConfig:
enabled: bool = True
batch_size: int = 100
flush_interval_seconds: float = 5.0
queue_capacity: int = 10000
redis_backup_enabled: bool = False
```

### Integration

- Add to `ai.backend.manager.config.unified.ManagerConfig`
- Initialize in manager server startup
- Graceful shutdown handling (flush remaining logs)

## Testing

- Unit tests for queue operations
- Unit tests for batch writer logic
- Integration tests with mock DB

## Acceptance Criteria

- Memory queue accepts logs without blocking
- Batch writer flushes on interval and threshold
- Clean shutdown without data loss
- All tests passing

JIRA Issue: BA-4233

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.