Per-LLM lifetime token limit
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Is this related to a problem? Please describe.
We want NVCF to track and limit a user's lifetime token usage (e.g. 5M tokens in/200k tokens out).
Describe the solution you'd like
I can define in the yaml tokenLifetimeLimit: 5000000
Describe alternatives you've considered
A longer time horizon of yearly rate limiting would also work.
Additional context
This is to curb explosive WCE invocations growth due to increase in new user sign ups on build.nvidia.com.
By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the YAML configuration handling and the existing usage or rate-limiting path. Work out how lifetime input and output token limits should be represented and enforced; the issue is complete when a configured limit such as 5M input and 200k output tokens is tracked and prevents further usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100