redpanda-data / redpanda-data/connect
Feature Request - 'limit' for sqlite buffer
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 8.8k
- Forks
- 969
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 64
Description
It would be helpful to have a 'limit' option for sqlite buffer like exists for the memory buffer.
Reference: https://www.benthos.dev/docs/components/buffers/memory#limit
This would help lower issues with running out of disk on the host system.
Products like logstash go as far to to print error when the host system does not have enough disk to support the disk limits on the queues.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names the sqlite buffer and the memory buffer's documented limit option; start by comparing those buffer implementations with the referenced documentation. No source file or test is named, so identify the existing sqlite and memory buffer entry points first; done means sqlite buffering supports the requested limit and addresses host disk exhaustion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sqlite
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100