Design batched logging architecture and interfaces
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Objective
Design the architecture and interfaces for the batched logging system before implementation.
## Tasks
1. **Define Core Interfaces**
- `LogQueueManager` - Manages memory queues for different log types
- `BatchWriter` - Handles periodic/threshold-based batch writes
- `LogBuffer` - Memory buffer with configurable capacity
1. **Define Configuration**
- Batch size threshold (default: 100 items)
- Flush interval (default: 5 seconds)
- Queue capacity limits
- Redis backup settings (per log type)
1. **Design Data Flow**
- Log ingestion path (non-blocking)
- Batch assembly logic
- Flush trigger conditions (time-based, size-based)
- Error handling and retry logic
1. **Integration Points**
- Repository layer changes (add bulk insert methods)
- Service layer changes (replace direct DB calls)
- Configuration management (unified config)
## Deliverables
- Architecture document or design diagrams
- Interface definitions (Protocol classes)
- Configuration schema
- Integration plan
## Acceptance Criteria
- Clear separation between log ingestion and DB writing
- Support for different log types with different policies
- Graceful handling of backpressure scenarios
- Design reviewed and approved
JIRA Issue: BA-4232
Contributor guide
Assessment
This issue has not been assessed yet.