google-gemini / google-gemini/gemini-fullstack-langgraph-quickstart

Feature Request: Add method to retrieve token count from LLM responses

Open
#177 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
18.3k
Forks
3.1k
PR merge metrics
No merged PRs in 30d

Description

### Description of the feature request:

Currently, when using the google-gemini/gemini-fullstack-langgraph-quickstart library, there isn't an apparent way to directly access the number of input or output tokens consumed by the LLM for a given prompt/response. It would be highly beneficial to expose this information.

I propose adding a method or attribute to the LLM response object, or to the invoke result, that allows users to easily retrieve the input_token_count and output_token_count.

### What problem are you trying to solve with this feature?

Without access to token counts, it's difficult to:

- Monitor API costs: Users cannot accurately track or estimate their Gemini API usage expenses.
- Implement rate limiting/budget management: For applications requiring strict control over LLM consumption, token count is essential.
- Optimize prompt engineering: Understanding token usage helps in refining prompts for efficiency.

### Any other information you'd like to share?

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.