ConduitIO / ConduitIO/conduit

pipeline.Service.UpdateStatus is not concurrency-safe for the same pipeline ID

Open
#2,579 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
610
Forks
63
Avg merge
12h 28m
Merged PRs (30d)
57

Description

Surfaced by the #2578 concurrency review. `pipeline.Service.UpdateStatus` writes `instance.Error = errMsg` **unlocked** (`pkg/pipeline/service.go:375`) and `store.Set` JSON-encodes the whole instance (reads all fields) **unlocked**; only `SetStatus` is `statusLock`-guarded (`instance.go:94`). Concurrent same-ID `UpdateStatus` calls therefore race on the `*pipeline.Instance` — under `-race` this is a real DATA RACE that can clobber a terminal status back to `Running`.

#2578 works around it caller-side in `pkg/lifecycle-poc` (a `startupDone` channel serializing the initial vs terminal write). The production `pkg/lifecycle` only avoids it *structurally* (it writes `Running` before registering the cleanup goroutine, and real nodes don't complete instantly) — so the shared type is still unsafe by construction.

**Fix at the right layer:** make `Instance` status + error + persist atomic under `statusLock` so the type is safe by construction, then drop the caller-side workaround. Tier 1 (data-path status integrity) — needs a race regression test.

Contributor guide

Open the contributing guide

Research direction

Read pkg/pipeline/service.go:375 and instance.go:94 to trace how UpdateStatus, SetStatus, and store.Set access the shared instance. Then inspect the startupDone workaround in pkg/lifecycle-poc and add the requested race regression test. Done means same-ID concurrent status updates are safe under -race and the caller-side workaround can be removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.