ConduitIO / ConduitIO/conduit

dev: harden conduit run --dev (debouncer growth, --json naming, human ConfigPath)

Open
#2,621 0 comments 0 reactions 0 assignees View on GitHub
housekeeping
Dominant language
Go
Stars
610
Forks
63
Avg merge
12h 28m
Merged PRs (30d)
57

Description

Follow-ups from the independent review of #2616 (`conduit run --dev` hot-reload). All non-blocking; grouped for tracking.

1. **Unbounded debouncer/goroutine growth per distinct filename.** `Watcher.consume` creates one debouncer goroutine + map entry per distinct matching filename, living until ctx cancel. `handleDeleted` frees the `filePipelines` entry but not the debouncer. Bounded in realistic dev use (only matching files), so not a practical leak — but a long dev session that churns many distinct pipeline filenames grows monotonically. Consider retiring a debouncer when its file is deleted (idle + no pending).

2. **`--dev.json` vs the repo-standard `--json`.** The dev event stream is surfaced as `--dev.json` rather than `--json`. Defensible for a scoped streaming surface on a long-running `run` command, but it diverges from the "every command supports `--json`" convention. Decide: keep and document the rationale, or alias `--json`.

3. **`renderHuman` omits `ConfigPath`.** `events.go` captures `ErrorInfo.ConfigPath` for `--dev.json` but the human renderer never prints it — an operator watching the text stream loses the failing config path. Cheap to add.

Source: `pkg/conduit/dev/{watcher.go,apply.go,events.go}`.

Contributor guide

Open the contributing guide

Research direction

Start with pkg/conduit/dev/watcher.go, apply.go, and events.go, focusing on Watcher.consume, handleDeleted, the dev JSON flag, and renderHuman. Decide how deleted-file debouncers and JSON naming should behave, then ensure the human event output includes ErrorInfo.ConfigPath. Done means the three review follow-ups have documented or implemented resolutions.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, devtools
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.