Support setting timeout on delete/update/patch stream API
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- go, kafka
- Domain
- api, stream-processing
Research direction
Start by tracing the delete, update, and patch stream API entry points and their existing graceful-shutdown path. Verify how a timeout query parameter should affect blocked streams, then confirm that all three operations force deletion after the timeout while preserving current behavior when it is absent.
Written by the indexing model from the issue text.
Description
Benthos streams mode provdes API to delete a stream. With current implementation, the API tries to shutdown the stream gracefully. However, it's possible that a stream is blocked. For example, the kafka and kafka_franz outputs, if a bad broker address is used, those outputs will keep retrying the connection error (for kafka, it retries in Connect, while kafka_franz retries in BatchWrite). In such case, the delete stream API will be blocked and the stream will never be deleted.
To overcome this problem, one solution is to add a timeout query parameter to the delete stream API, e.g. DELETE /streams/{id}?timeout=<duration>. If the timeout query is provided, the API will force deleting the stream without waiting any longer. If it's not provided, the API will behave as it does today.
Since update/patch stream API also delete streams, this change should also apply to them as well.
- Dominant language
- Go
- Stars
- 8.8k
- Forks
- 969
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 64
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from redpanda-data/connect
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
redpanda-data/connect#4837 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
redpanda-data/connect#4711 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
redpanda-data/connect#4705 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
redpanda-data/connect#4515 · 1 reaction ·
-
needs investigation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
redpanda-data/connect#4511 · 1 comment ·
All issues in redpanda-data/connect
Similar issues
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-cao#13143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
yanet-platform/ipfw-go#129 ·
-
bug confmap/provider/googlesecretmanagerprovider needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-telemetry/opentelemetry-collector-contrib#51273 · 2 comments ·
-
bug: AI Gateway client filter lists "Unknown" twice when NULL and literal Unknown clients coexist Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100