clj-commons / clj-commons/durable-queue
Thread safety of take!, put!
- Dominant language
- Clojure
- Stars
- 405
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
Should we consider put! and take! threadsafe?
I have one thread doing puts in bursts at regular interval, and N workers on their own respective threads consuming the queue. Sometimes I get into a situation where I have 1 in progress task and all other workers waiting on tasks while the queue keeps growing (essentially deadlocked at this point). In the code I take!, deref and mark completed tasks immediately (did that to exclude a bug on complete!).
Should I serialize the takes or is this a potential bug? I ll try to come up with a repro at work tomorrow.
I also get the occasional negative in-progress while testing:
`09:22:14.938 [clojure-agent-send-off-pool-0] INFO sink.testdq - {foo {:num-slabs 1, :num-active-slabs 1, :enqueued 10947, :retried 0, :completed 10947, :in-progress -1}}`
Thanks for the lib by the way, it s super useful.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.