attharmirza / attharmirza/census-questions-api
Make more efficient use of tokens
- Dominant language
- JavaScript
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Right now each call to Gemini takes into account the _entire_ function call, which includes my super long function call variables generated from all the census API variables. That means over 30,500 tokens per call. We only get 1 milllion per minute.
I think I can get rid of at least half of those tokens by asking Gemini to pull state/county/city names from the natural language prompt (rather than converting those to FIPS codes), and then using geocoding or something like that to get FIPS codes. Beyond that, we have to get clever.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing each Gemini call and the generated Census API function-call variables described in the issue. Measure the current token use, then evaluate extracting state, county, and city names from the natural-language prompt before geocoding them to FIPS codes. Done means substantially lower per-call token usage without losing Census data retrieval.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- ai, api, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100