[Feature]: Ability to compare private branch perf to production, and stop storing trtllm private branch experiments in prod

🚀 The feature, motivation and pitch

1. Currently when running a private branch of the dashboard the results are not stored in the prod DB so the regression chart won't show it. we need to be able to compare results of a private branch to production.
2. we want to compare a trtllm commit/branch to the production results. currently when setting TRTLLM_PRECOMPILED_COMMIT it will store the results to the prod DB, so we are able to compare them, however storing the results to prod is not desired and will make it hard to detect regressions on main. so the results should be stored to dev DB, and yet we should be able to compare them to the prod results.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.