scylladb / scylladb/scylla-cdc-java

Potential race condition while closing Worker

Open
#65 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

base
Dominant language
Java
Stars
32
Forks
23
Avg merge
2d 6h
Merged PRs (30d)
3

Description

As observed in a CI run (but very hard to reproduce, this run somehow reused a CI container from a previous run which might have polluted the environment): https://github.com/scylladb/scylla-cdc-java/runs/3477278901

The issue seems to arise when two threads concurrently run Worker::stop() and Worker::makeCallable() "continuation":

Thread A:
https://github.com/scylladb/scylla-cdc-java/blob/ab828eb0305171ffa48d6114dc73aaba0257ffdc/scylla-cdc-base/src/main/java/com/scylladb/cdc/model/worker/Worker.java#L153-L156

Thread B:
https://github.com/scylladb/scylla-cdc-java/blob/ab828eb0305171ffa48d6114dc73aaba0257ffdc/scylla-cdc-base/src/main/java/com/scylladb/cdc/model/worker/Worker.java#L114-L123

  1. Thread B on line 118 checks shouldStop() and proceeds to go to else if body.
  2. Thread A runs line 154 and 155, setting shouldStop=true and shutting down the executor service.
  3. Thread B on line 119 tries to submit a task to executor service, which ends with java.util.concurrent.RejectedExecutionException as the executor service is now terminated.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in scylla-cdc-base/src/main/java/com/scylladb/cdc/model/worker/Worker.java at the referenced lines in stop() and makeCallable(). Trace the concurrent executor shutdown and task submission paths, then verify that concurrent calls do not produce RejectedExecutionException while closing the Worker.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.