google-gemini / google-gemini/gemini-cli
Feature Request: Improve 429 (Rate Limit) UX via CLI-Level "Cool-Down" Wrapper & Terminal Countdown
- Dominant language
- TypeScript
- Stars
- 107k
- Forks
- 14.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 45
Description
### What would you like to be added?
## Context & Background
In enterprise, state government, or proxy-restricted networks, developers often operate under strict quota ceilings (such as Google Cloud Vertex AI rate limits) and sliding-window rate limiters. In these environments, `429 Too Many Requests` errors are common.
## The Problem: "Silent Hangs" and Quota Hammering
Currently, the Gemini CLI delegates retry logic directly to the underlying low-level Google GenAI SDK.
While this is standard, it creates two major user experience issues in terminal environments:
1. **Unresponsive Terminal ("Silent Hang")**: The SDK handles retry attempts silently over HTTP with exponential backoff. Because there are no intermediate events or hooks, the CLI remains frozen with a blinking cursor or spinner. To the user, a 1-to-2 minute silent wait looks identical to a crashed or hung process, leading them to force-abort (`Ctrl + C` or `Esc`).
2. **Quota Extension via "Hammering"**: By defaulting to `10` maximum attempts, the silent SDK continues to slam the API in rapid succession. Under a sliding-window rate-limiting model, continuous requests from a blocked client can actually extend the duration of the lockout, while simultaneously flooding terminal logs with hundreds of redundant `429` errors.
3. **Manual Recovery Paradox**: Frequently, when a network request hangs or hits a rate limit, manually aborting the command (pressing `Esc` or `Ctrl + C`) and immediately re-running it succeeds instantly. This is because the manual retry "warm-boots" the proxy or gateway connection, making the CLI's silent wait even more redundant.
---
## Proposed Solution: CLI-Level Fail-Fast & Countdown Wrapper
Instead of letting the low-level Google SDK retry silently in secret, we propose **disabling SDK-level retries (`maxRetries: 0`) and managing retry policies directly at the CLI application level.**
### 1. Intercept 429 Errors
When the CLI wrapper intercepts a `429` response, it should suspend the request and print a clear, user-friendly notification in the terminal.
### 2. Display a Visual Countdown Timer
Rather than sitting in a silent block, the terminal should render an active "cooling down" countdown. For example:
```text
⚠️ [429 Rate Limit Exceeded] API quota limit reached.
Cooling down... Retrying in 12s... (Attempt 1/3)
```
*(The countdown should update in-place on the terminal line, showing active progress.)*
### 3. Implement Fail-Fast and Configurable Max Attempts
Add a configurable setting (e.g., `"general.maxAttempts": 3` by default) so that if a rate limit or network freeze is solid, the CLI fails fast and gracefully returns control to the user, instead of looping indefinitely.
---
## Why This is a Massive Win for Developers:
- **Clarity and Confidence**: The user knows exactly what is happening (rate limit) and when the program will try again, completely eliminating "did it crash?" anxiety.
- **Quota-Friendly**: Pausing for a dedicated 10-to-15 second countdown gives sliding-window rate limiters proper time to "cool down" and clear, resulting in fewer total retries and faster overall resolution.
- **Logger Cleanliness**: Eliminates the endless walls of redundant `429` error logs.
---
## Suggested Implementation Outline (TypeScript / Node.js)
```typescript
async function callModelWithCoolDown(prompt, maxAttempts = 3) {
let attempts = 0;
while (attempts < maxAttempts) {
try {
// Disable low-level SDK silent retries
return await googleSdk.generateContent(prompt, { maxRetries: 0 });
} catch (error) {
if (error.status === 429) {
attempts++;
if (attempts >= maxAttempts) {
throw new Error("API Rate Limit Exceeded. Please try again in a moment.");
}
// Render in-place countdown in terminal
for (let i = 10; i > 0; i--) {
process.stdout.write(`\r⚠️ [429 Rate Limit] Quota reached. Cooling down... Retrying in ${i}s... (Attempt ${attempts}/${maxAttempts})`);
await sleep(1000);
}
process.stdout.write('\n');
} else {
throw error; // Fail immediately on non-429 errors
}
}
}
}
```
### Why is this needed?
Currently, when a 429 rate limit or network stall occurs, the low-level Google SDK retries silently under the hood. This causes the CLI to appear to "freeze" or "hang" in the terminal for several minutes without any visual feedback.
This creates a frustrating user experience because developers cannot tell if the CLI has crashed or is actually working, which usually leads them to manually force-abort the session. Furthermore, rapidly hammering a rate-limited endpoint 10 times in a row can actually extend the duration of the quota lockout on sliding-window rate limiters, while flooding terminal logs with redundant 429 errors.
Moving the retry and backoff logic to the CLI level—and showing an active visual countdown timer—completely eliminates "silent hang" anxiety, protects API quotas from being hammered while blocked, and significantly improves terminal responsiveness on strict enterprise or restricted networks.
### Additional context
- This issue is highly prominent when running the CLI inside proxied corporate or state government networks, where connections are subject to network handshake stalls and low Vertex AI quota ceilings.
- We observed that when a request hangs or hits a rate limit, manually aborting it (pressing `Esc` or `Ctrl + C`) and immediately re-running it often succeeds instantly because the manual restart "warm-boots" the proxy connection. A CLI fail-fast setting (such as a default of `maxAttempts: 3`) mimics this behavior automatically.
### Suggested Implementation Blueprint (TypeScript/Node.js)
By configuring the low-level Google SDK to turn off silent retries (`maxRetries: 0`), the CLI can intercept 429s and render a clean, in-place countdown:
```typescript
async function callModelWithCoolDown(prompt, maxAttempts = 3) {
let attempts = 0;
while (attempts < maxAttempts) {
try {
// Disable low-level SDK silent retries
return await googleSdk.generateContent(prompt, { maxRetries: 0 });
} catch (error) {
if (error.status === 429) {
attempts++;
if (attempts >= maxAttempts) {
throw new Error("API Rate Limit Exceeded. Please try again in a moment.");
}
// Render in-place countdown in terminal
for (let i = 10; i > 0; i--) {
process.stdout.write(\r⚠️ [429 Rate Limit] Quota reached. Cooling down... Retrying in ${i}s... (Attempt ${attempts}/${maxAttempts}));
await sleep(1000);
}
process.stdout.write('\n');
} else {
throw error; // Fail immediately on non-429 errors
}
}
}
}
```
---
*Note: This feature request was drafted in collaboration with Gemini CLI during a peer-programming session, based on direct terminal diagnostics and real-world observations of rate-limiting behaviors on restricted enterprise networks.*
Contributor guide
Research direction
Locate the CLI's Google GenAI request path, terminal output or rendering code, and general settings handling; begin by tracing how 429 responses and SDK retries currently surface. Check existing retry, cancellation, and terminal-output tests if present. Done means rate limits provide visible countdown feedback, honor a configurable attempt limit, and return control cleanly after the limit is reached.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, node.js, typescript
- Domain
- api, cli, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100