attharmirza / attharmirza/census-questions-api

Make more efficient use of tokens

Open
#3 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
JavaScript
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Right now each call to Gemini takes into account the _entire_ function call, which includes my super long function call variables generated from all the census API variables. That means over 30,500 tokens per call. We only get 1 milllion per minute.

I think I can get rid of at least half of those tokens by asking Gemini to pull state/county/city names from the natural language prompt (rather than converting those to FIPS codes), and then using geocoding or something like that to get FIPS codes. Beyond that, we have to get clever.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing each Gemini call and the generated Census API function-call variables described in the issue. Measure the current token use, then evaluate extracting state, county, and city names from the natural-language prompt before geocoding them to FIPS codes. Done means substantially lower per-call token usage without losing Census data retrieval.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, api, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.