Kong / Kong/developer.konghq.com

[3.13] AI Gateway: Document support for a basic cross-LLM tokenizer for token rate-limiting

Open
#3,358 0 comments 0 reactions 0 assignees View on GitHub
internal release-docs
Dominant language
Ruby
Stars
28
Forks
121
Avg merge
1d 4h
Merged PRs (30d)
313

Description

## Description

Add documentation for a basic tokenizer option that approximates token counts across different LLMs, enabling pre-request token estimation in the token rate-limiting plugin. This is to help users predict and enforce token budgets before sending a request to the model, without depending on provider-specific tokenizers.

## Definition of Done

[TBC]

* Update token rate-limiting plugin docs to describe the basic tokenizer option
* Provide guidance on expected accuracy and use cases
* Add configuration example showing how to enable it
* Add brief how-to for estimating and enforcing token limits before request execution

## Information

Related ticket: https://konghq.atlassian.net/browse/AG-527

### Size

S

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the token rate-limiting plugin documentation and review the existing configuration examples and tokenizer options. Document the basic cross-LLM tokenizer, its expected accuracy and use cases, and how to estimate and enforce limits before requests execute. Confirm the final scope against related ticket AG-527, since the Definition of Done is still marked TBC.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.