redpanda-data / redpanda-data/connect
Deleting a stream via the REST API can block forever
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 8.8k
- Forks
- 969
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 64
Description
Steps to reproduce:
- Start Connect:
> redpanda-connect streams
- Create a simple stream via the REST API with a failing output which blocks messages forever:
> curl http://localhost:4195/streams/bar -X POST --data-binary @- <<EOF
input:
broker:
inputs:
- generate:
count: 1
interval: 1s
mapping: root = ""
output:
retry:
output:
http_client:
url: localhost:8080
EOF
- Try to delete the stream via the REST API:
> curl http://localhost:4195/streams/bar -X DELETE
Note that the curl command will hang and cancelling it via Ctrl+C will produce the following in the Connect log:
WARN Failed to send message: localhost:8080: Post "localhost:8080": unsupported protocol scheme "localhost" @service=benthos label="" path=root.output stream=bar
ERRO Encountered error whilst attempting to shut down gracefully: context canceled @service=benthos stream=bar
INFO Some components prevented forced termination as they were either blocked from delivering data or from acknowledging delivered data within the shutdown timeout. This could potentially cause duplicate messages to be delivered on the next run. @service=benthos stream=bar
ERRO Streams CRUD Error: context canceled @service=benthos
No more logs are produced after this, but trying to recreate the stream returns an error message saying "Stream already exists". Running the DELETE command again succeeds and now the stream can be recreated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the redpanda-connect streams command and the REST API POST and DELETE requests shown above, then trace the stream deletion and graceful shutdown path. Done means deleting a stream with a blocked failing output returns instead of hanging, the stream is no longer reported as existing, and it can be recreated successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100