plugins: a `grpc` plugin — one unary call against reflection or a pinned descriptor set, the `buf curl` shape the ecosystem has no answer for
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 9
- Forks
- 0
- Avg merge
- 3h 3m
- Merged PRs (30d)
- 509
Description
Deferred from #2021, which landed six plugins (oci, scim, ssh, docker, jose, oidc) and deliberately left this one out.
Problem
A workflow that has to call a gRPC service today has no way to do it. http speaks HTTP/1.1 and JSON; gRPC is HTTP/2 with length-prefixed protobuf framing, and the message a service takes is a schema the workflow does not hold. Every deployment that runs Connect or gRPC services internally — which is the shape this repository itself is built in — reaches them from a Flowfile only by putting a shell script in front of them.
buf curl is the reference for what this should feel like: name a service and method, supply the request as JSON, get the response as JSON, with the schema resolved either from the server's reflection service or from a descriptor set the operator pinned.
Shape
One task, grpc.call, unary only:
- Schema from one of two places. Server reflection (
grpc.reflection.v1.ServerReflection), or aFileDescriptorSetan operator configured — the pinned form being the one a deployment uses when it does not want the server deciding what its own messages mean. A workflow supplies neither. - Unary only. Streaming has no bounded result to put in durable history, and a step that returns "the first N messages of a stream" is a reporting limit standing in for a work limit (invariant 5).
- Bounded in every direction: request size, response size derived from
flowstatev1.MaxTaskOutputBytesthe way #2021'sdockerandociceilings now are, descriptor-set size, reflection-response size, call deadline. - The deployment's egress policy governs the dial, per
plugins/sshandplugins/docker's remote daemon in #2021 — which may mean agrpcscheme besidesshandpostgresinnetpolicy, or reusinghttpssince the transport is HTTP/2 over TLS. Worth deciding rather than assuming. - Credentials through the secret boundary, never a header a Flowfile composes.
Open questions
- Reflection is a capability: a server that offers it will describe every service it has. Is naming the method enough of a bound, or does the operator grant a service allowlist the way
sshgrants commands? - Connect, gRPC and gRPC-Web are three wire formats over the same schema. One task with a protocol input, or the Connect client's own negotiation?
- Does the descriptor set belong in the operator's environment (
--plugin-env grpc=..., the surface #2021 added) or in a grant file likedocker's andssh's?
Acceptance
grpc.callwith reflection and with a pinned descriptor set, both covered by areachable/test that launches the real binary against a real in-process gRPC server rather than a stub at the client boundary.- A worked example under
examples/plugins/grpc/that the corpus validates, listed inexamples/README.md. - Every bound stated in the plugin's
doc.goand asserted in a test, including the negative direction. - Egress governed on the actual dial path, with a test proving a denied destination is never reached.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing plugins/ssh and plugins/docker behavior, the netpolicy surface, and the changes from #2021. Resolve the open questions around protocol, reflection bounds, credentials, and descriptor-set configuration before implementation. Done means reachable/ tests cover reflection and pinned descriptors against a real gRPC server, examples/plugins/grpc/ is validated and listed, doc.go states every bound, and denied egress is tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- backend-api-design, documentation, networking, security, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100