[RFC]: Add openengine gRPC server support to trtllm-serve

[RFC]: Add openengine gRPC server support to trtllm-serve

Goal: Add openengine as a selectable gRPC protocol for trtllm-serve. Ship the protocol bindings as an opt-in pip extra so the default install gains no new dependency.

Motivation

OpenEngine is a vendor-neutral gRPC contract for inference engines being developed by the Dynamo team. Adding OpenEngine support to trtllm-serve is intended to allow it to serve as a drop-in inference engine for clients implementing the OpenEngine contract (for example, with the Dynamo sidecar architecture).

Dependency Impact

OpenEngine’s bindings are generated by buf and published on the Buf Schema Registry. Consume them as pip packages from the https://buf.build/gen/python index, pinned to OpenEngine v0.1.0 at BSR module commit 768a93c7b44e40f28c692ad0b471a8f2. Ship them as an optional extra: pip install --extra-index-url https://buf.build/gen/python "tensorrt_llm[openengine]". The extra is listed in requirements-openengine.txt and wired into setup.py extras_require["openengine"]. The default install gets no new dependency. No buf CLI, no vendored .proto, no committed generated protobuf bindings, no build-time codegen.

The extra pins:

• openengine-openengine-grpc-python — BSR-generated gRPC stubs
• openengine-openengine-protocolbuffers-python and -pyi — BSR-generated messages and type stubs
• grpcio>=1.67.1,<2
• protobuf>=6.31.1,<7

Proposed Change

• Add selector --grpc-protocol {smg,openengine}, default smg. --grpc is unchanged. Backward compatible.
• Add subpackage tensorrt_llm/grpc/openengine/
• Add test tests/unittest/grpc/openengine
• The default HTTP server is unchanged; the sibling is disabled unless the port is given.
• First integration is a protocol stub: every RPC returns UNIMPLEMENTED
• Will keep adding implementing new features sequentially..

Feedback Period

~1 week.

CC

@NVIDIA/trt-llm-runtime-devs — owns tensorrt_llm/commands/serve.py and tensorrt_llm/grpc.
@NVIDIA/trt-llm-oss-compliance — owns requirements.txt; dependency and license review.
Assignee: laikhtewari.

Any Other Things

OpenEngine is experimental and pre-adoption (v0.1.0). Breaking changes are possible until OpenEngine reaches a stable state. This lands behind an opt-in flag and an opt-in extra, so the blast radius is small. The OpenEngine community will be responsible for maintaining these changes within the TensorRT-LLM OpenEngine gRPC server.