Feature: Introduce Node.js Language Bindings for TRT-Model-Connect (Prototype)

Request type

New model or checkpoint support

Problem and use case

Description

Following our conversation in Discussion #967 regarding the exploration of new language bindings at runtime, I have created a prototype/Proof-of-Concept (PoC) for exposing the trtmc C++ backend to Node.js.

Currently, TRT-Model-Connect operates with highly optimized C++ and Python bindings. However, web servers and embedded AI applications heavily rely on the JavaScript/Node.js ecosystem. By introducing a Node.js native addon, developers can seamlessly load .bundle files and run LLM inference directly from their backend services.

Proposed outcome

What is included in this Prototype?

This PoC uses node-addon-api (N-API) to wrap the core C++ Pipeline class and expose it to JavaScript. The implementation is isolated in a new bindings/nodejs directory.

• PipelineWrapper: A C++ N-API class that holds a std::unique_ptr<trtmc::Pipeline>.
• load() Binding: Exposes trtmc::load to initialize a pipeline from a bundle path.
• generate() Binding: Accepts a text prompt and JSON config from JavaScript, executes the C++ inference, and returns a structured JavaScript object containing the generated text, token_ids, and timing metrics (prefill_ms, decode_ms).
• Build Infrastructure: A binding.gyp file configured to link against the generated tensorrt_model_connect library.

Example Usage

```javascript
const trtmc = require('./build/Release/trtmc_node.node');

// Load the model
const pipe = trtmc.load("model.bundle");

// Run inference
const result = pipe.generate("Hello, how are you?", {
max_new_tokens: 50,
temperature: 0.7
});

console.log(result.text);
```

Next Steps / Questions for Maintainers

1. Directory Structure: The prototype is currently placed in bindings/nodejs/. Does this align with the repository's long-term architectural goals?
2. Build System Integration: Currently relying on node-gyp. Should we migrate the Node.js build step into the root CMakeLists.txt using tools like cmake-js for a unified build pipeline?
3. Scalability across model APIs: As discussed, we need to ensure this wrapper pattern remains scalable across different model families without breaking isolation.

I will be opening a draft PR shortly with this prototype code so we can review the architecture and iterate on it!

Model and target details

No response

Alternatives considered

No response

Additional context

No response

Submission checks

• I searched open and closed issues and found no duplicate request.
• I removed secrets, private/internal evidence, personal paths, and restricted artifacts.

Start with Discussion #967 and the proposed prototype in bindings/nodejs, then review binding.gyp alongside the root CMakeLists.txt. The issue is ready only after maintainers decide the directory and build-system direction and the Node.js binding approach is reviewed in the planned draft PR.