temporalio / temporalio/sdk-typescript
Add @bufbuild/protobuf (protobuf-es) payload converters
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 917
- Forks
- 224
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 43
Description
Is your feature request related to a problem? Please describe.
The SDK ships payload converters for protobufjs only (@temporalio/common/lib/protobufs). Users who generate their proto types with @bufbuild/protobuf v2 can't pass those types to a Temporal client or workflow without writing their own payload converter. The two libraries produce incompatible message shapes (Schema + create(Schema, init) vs Message.create(init)), so the existing converters can't be reused.
Describe the solution you'd like
A parallel converter trio under @temporalio/common/lib/protobufs-es, mirroring the existing …/protobufs exports:
ProtobufEsBinaryPayloadConverterProtobufEsJsonPayloadConverterDefaultPayloadConverterWithProtobufsEs
Today — every protobuf-es user writes their own (~150 LOC) converter:
// payload-converter.ts
import { create, toBinary, fromBinary, toJson, fromJson, createRegistry } from '@bufbuild/protobuf';
// ...custom Binary + JSON converters, composite ordering,
// messageType metadata, Registry forwarding for google.protobuf.Any...
Proposed — first-class support, matching the protobufjs ergonomics:
// payload-converter.ts
import { createRegistry } from '@bufbuild/protobuf';
import { DefaultPayloadConverterWithProtobufsEs } from '@temporalio/common/lib/protobufs-es';
import { MyMessageSchema } from './gen/messages_pb';
export const payloadConverter = new DefaultPayloadConverterWithProtobufsEs({
registry: createRegistry(MyMessageSchema),
});
Sketch of the desired properties:
@bufbuild/protobufas an optional peer dependency so users on the protobufjs path pay nothing.- Wire-compatible with the existing protobufjs converters:
- A payload produced by either converter (binary or JSON) round-trips correctly through the other.
- Same
encodingmarkers (binary/protobuf,json/protobuf) and fully-qualifiedmessageTypemetadata. - Composite ordering matches
DefaultPayloadConverterWithProtobufs(Undefined → Binary → ProtobufJson → ProtobufBinary → Json), so on-the-wire converter selection is identical.
- Constructor accepts either a pre-built
Registryor a schema array (callscreateRegistryinternally).
Additional context
The existing docs/protobuf-libraries.md is the SDK's original (Dec '21) decision log for protobuf library choice. It defines three criteria — (A) TS types, (B) runtime-checkable message instances, (C) spec-compliant proto3 JSON — and notes that protobufjs scores A+B but not C, which is why the SDK bolts on proto3-json-serializer and patchProtobufRoot. @bufbuild/protobuf v2 satisfies A+B+C natively, and its generated output addresses every item in that doc's "Future work" section (spec-compliant to/fromJSON, fully-qualified typename, "this is a generated file" header).
Concrete use case driving this
I'm building protoc-gen-ts-temporal and protoc-gen-rust-temporal (TypeScript and Rust analogues of protoc-gen-go-temporal) so that a single .proto file generates type-safe Temporal clients, workers, and activity stubs across languages.
I'm trying to build a Node service that accepts a webhook and starts a workflow whose worker is implemented in Rust, with no intermediate Rust HTTP server that exists only to translate the webhook into a workflow start. For that to work, the TS client and the Rust worker have to agree on the wire format of the workflow args. protoc-gen-ts-temporal is built on @bufbuild/protoc-gen-es, so its generated message types are @bufbuild/protobuf v2.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the existing converters and exports under @temporalio/common/lib/protobufs, then read docs/protobuf-libraries.md for the SDK's compatibility criteria. Add the parallel protobuf-es converter exports with optional dependency and registry support, and verify binary/JSON interoperability, metadata, and composite ordering against the existing protobufjs behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend-api-design, developer-experience
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100