ConduitIO / ConduitIO/conduit

pipelines inspect: --records N in-flight sample via InspectConnector stream

Open
#2,589 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
610
Forks
63
Avg merge
12h 28m
Merged PRs (30d)
57

Description

From PR #2587 (inspect). The v1 inspect ships the live status + per-stage view. The design's `--records N` bounded in-flight record sample (over the `InspectConnector`/`InspectProcessor` server-streams, backed by `pkg/inspector`) is deferred: it needs `InspectConnector` added to the CLI's `ConnectorService` interface + bounded stream handling + a running-pipeline-with-data integration test. Tier 2.

Contributor guide

Open the contributing guide

Research direction

Start with PR #2587 and the v1 inspect implementation, then trace the CLI's ConnectorService interface to the InspectConnector and InspectProcessor server streams backed by pkg/inspector. Add bounded handling for --records N and a running-pipeline-with-data integration test; done means the CLI returns the requested in-flight sample without unbounded streaming.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.