ConduitIO / ConduitIO/conduit

deploy/apply: live-server RPC for standalone running-pipeline safety

Open
#2,588 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
610
Forks
63
Avg merge
12h 28m
Merged PRs (30d)
57

Description

From PR #2586 (deploy/apply, Tier 1). Standalone `apply` can't positively confirm a pipeline is running, because `pipeline.Service.Init` launders persisted `StatusRunning`→`StatusSystemStopped` (it can't know if the process that set it is alive). Wave-2 mitigation: `NewLocalService` is default-deny — only BadgerDB is allowed, because its exclusive store-directory lock makes a concurrent live `conduit run` fail `OpenStore` closed; SQLite/Postgres/InMemory are refused (`provisioning.standalone_unsafe`).

The real fix: when a Conduit server is running, `deploy/apply` should reach it over an RPC so the running-pipeline check is answered by the process that actually owns the pipeline (and apply flows through the live server's stop-drain-restart). That lifts the Badger-only restriction and closes the gap for every store type. Tier 1.

Contributor guide

Open the contributing guide

Research direction

Start by reading PR #2586, deploy/apply, pipeline.Service.Init, and NewLocalService to understand the current standalone safety behavior. Trace how a running server could answer the pipeline-running check over RPC and how apply should use its stop-drain-restart flow; done means the Badger-only restriction is lifted safely for SQLite, Postgres, and InMemory.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.