Kong / Kong/developer.konghq.com
[3.13] AI Gateway: Document support for a basic cross-LLM tokenizer for token rate-limiting
- Dominant language
- Ruby
- Stars
- 28
- Forks
- 121
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 313
Description
## Description
Add documentation for a basic tokenizer option that approximates token counts across different LLMs, enabling pre-request token estimation in the token rate-limiting plugin. This is to help users predict and enforce token budgets before sending a request to the model, without depending on provider-specific tokenizers.
## Definition of Done
[TBC]
* Update token rate-limiting plugin docs to describe the basic tokenizer option
* Provide guidance on expected accuracy and use cases
* Add configuration example showing how to enable it
* Add brief how-to for estimating and enforcing token limits before request execution
## Information
Related ticket: https://konghq.atlassian.net/browse/AG-527
### Size
S
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the token rate-limiting plugin documentation and review the existing configuration examples and tokenizer options. Document the basic cross-LLM tokenizer, its expected accuracy and use cases, and how to estimate and enforce limits before requests execute. Confirm the final scope against related ticket AG-527, since the Definition of Done is still marked TBC.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100