in_kubernetes_events: add SQLite indexes for event deduplication and cleanup
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 58
Description
**Is your feature request related to a problem? Please describe.**
With `db` enabled, `in_kubernetes_events` performs per-event deduplication by `uid` and cleanup by `created`. Neither column is indexed, so both queries scan the SQLite table.
With about 91,000 Kubernetes Event objects, this caused sustained CPU usage of one core during relisting. After the watch stabilized, CPU remained around 200m.
**Describe the solution you'd like**
Create indexes for new and existing databases:
```sql
CREATE INDEX IF NOT EXISTS idx_in_kubernetes_events_uid
ON in_kubernetes_events(uid);
CREATE INDEX IF NOT EXISTS idx_in_kubernetes_events_created
ON in_kubernetes_events(created);
```
**Describe alternatives you've considered**
- Removing `db` avoids the scans, but loses persistent deduplication across restarts.
- Creating the indexes manually works, but must be repeated if the database is recreated.
**Additional context**
Creating both indexes manually reduced steady CPU from about 200m to 20m without restarting Fluent Bit or changing its configuration.
The missing `uid` index was mentioned in #9787, but #9894 only fixed the cleanup timestamp calculation. The indexes are still absent from the current schema.
Contributor guide
Research direction
Start at the in_kubernetes_events schema and its database initialization or migration entry points, then inspect the uid deduplication and created cleanup queries. Verify the requested indexes are created for both new and existing databases and confirm the relevant queries use them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, kubernetes, sqlite
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100