[RFC]: User Telemetry in TensorRT LLM

Motivation.

The trtllm team is considering adding anonymous usage data collection, to help us better understand what parts of the product are used/how they're used to inform future prioritization.

Proposed Change.

It would be fairly similar to what vllm describes here: (https://docs.vllm.ai/en/stable/usage/usage_stats/).
It would be opt-outable by CLI, config, or env variable. The data would be aggregated locally on some time scale (~10 minutes) before being sent to the telemetry endpoint. The data types that have been discussed are:

• uptime of deployment
• number of GPUs/nodes
• GPU type
• model architecture if it's a known one
• ingress point into trtllm (ex LLM API vs CLI)
• features used (quantization, parallelism, speculative decoding, P-D Dissag, LoRA)

Some important aspects:

• No identifying information, persistent identifiers, prompts, or model outputs will be collected
• Easy opt-out
• CI enforcement to avoid unintentional changes to data collection

I'm wondering what initial impressions of this are:

• Is it something that you think you would leave on, opt-out but not mind that it was added, or would it be problematic even if you opt out?
• Are there any of the data types that you would prefer not be collected or any others you would want the trtllm team to have insight into?
• Any other thoughts/questions/concerns

Thanks for your thoughts on this!

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.