Migrate audit log to batched logging system
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
## Objective
Migrate audit log creation from direct DB writes to the batched logging system.
## Implementation Details
### Repository Changes
1. **Update** `AuditLogDBSource`
- Add `bulk_create()` method using `BulkCreator`
- Keep existing `create()` for backward compatibility
```python
async def bulk_create(
self,
creators: Sequence[Creator[AuditLogRow]]
) -> list[AuditLogData]:
async with self._db.begin_session() as db_sess:
result = await execute_bulk_creator(db_sess, BulkCreator(specs=[c.spec for c in creators]))
return [row.to_dataclass() for row in result.rows]
```
1. **Update** `AuditLogRepository`
- Route `create()` calls to queue manager
- Add `flush()` method for manual flush operations
### Service Changes
1. **Update** `AuditLogService`
- Change `create()` to enqueue instead of direct DB write
- Return immediately without waiting for DB write
### Integration
- Wire LogQueueManager into AuditLogService
- Configure batch writer for audit logs
- Set Redis backup enabled for audit logs (critical)
## Testing
- Test audit log enqueueing
- Test batch creation with multiple entries
- Test flush on interval and threshold
- Integration tests with real DB
## Migration Strategy
- Feature flag for gradual rollout
- Monitor queue depth and flush latency
- Rollback plan if issues detected
## Acceptance Criteria
- Audit logs written in batches
- No functional regression
- Reduced DB transaction count (target: 90%)
- All tests passing
JIRA Issue: BA-4234
Contributor guide
Research direction
Start by locating AuditLogDBSource, AuditLogRepository, AuditLogService, BulkCreator, and LogQueueManager, then trace the existing audit-log creation path. Review the queue and batch-writer configuration, including Redis backup and the feature flag. Done means batched writes, enqueue and flush behavior, integration coverage, and the stated transaction-reduction target without regression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- backend, databases, devops
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100