google / google/langextract

Feature Request: Token Usage Tracking (or model metadata?) Support

Open
#475 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
38.6k
Forks
2.7k
Avg merge
3d 15h
Merged PRs (30d)
3

Description

Related to #425, #263

Is there a consensus on implementing token usage tracking?

The most bare-bones option would probably be to just add a `get_usage()` method to [BaseLanguageModel](https://github.com/google/langextract/blob/fef3e7db723e87d9cdd11dfeda219bf4fa269350/langextract/core/base_model.py#L31) which returns the accumulated token usage of that model. Then that would be implemented for the supported models and a user could then call that method with a model from [factory.create_model](https://github.com/google/langextract/blob/fef3e7db723e87d9cdd11dfeda219bf4fa269350/langextract/factory.py#L103).

But maybe it'd be better to have a more general `get_metadata()` method for `BaseLanguageModel` that happens to return usage as part of it for the supported models.

For the general metadata option, maybe the output looks something like
```
{
usage: {
intput_tokens: int,
output_tokens: int
}
}
```

I'd be happy to contribute to this feature myself.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.