Implement memory queue and batch writer foundation
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Objective
Implement the core infrastructure for batched logging: memory queue management and batch writer.
## Implementation Details
### Components to Create
1. `LogQueueManager` (`src/ai/backend/manager/logging/queue_manager.py`)
- Manages asyncio.Queue instances for each log type
- Provides non-blocking put operations
- Handles queue capacity monitoring
1. `BatchWriter` (`src/ai/backend/manager/logging/batch_writer.py`)
- Background task consuming from queues
- Periodic flush (configurable interval)
- Threshold-based flush (configurable size)
- Uses BulkCreator pattern for DB writes
1. `LogBuffer` (`src/ai/backend/manager/logging/buffer.py`)
- In-memory accumulator for log entries
- Thread-safe operations
- Capacity management
### Configuration
```python
@dataclass
class BatchLoggingConfig:
enabled: bool = True
batch_size: int = 100
flush_interval_seconds: float = 5.0
queue_capacity: int = 10000
redis_backup_enabled: bool = False
```
### Integration
- Add to `ai.backend.manager.config.unified.ManagerConfig`
- Initialize in manager server startup
- Graceful shutdown handling (flush remaining logs)
## Testing
- Unit tests for queue operations
- Unit tests for batch writer logic
- Integration tests with mock DB
## Acceptance Criteria
- Memory queue accepts logs without blocking
- Batch writer flushes on interval and threshold
- Clean shutdown without data loss
- All tests passing
JIRA Issue: BA-4233
Contributor guide
Assessment
This issue has not been assessed yet.