[Bug]: Profile pre-flight validation reports budget-exceeded keys as valid and blocks ~2 min on retries

Is there an existing issue for the same bug?

• I have searched existing issues and this is not a duplicate.

Bug Description

POST /api/profiles/{name}/validate (LLM profile pre-flight, openhands-agent-server/openhands/agent_server/profiles_router.py) has two problems:

1. False positive. _TRANSIENT_ERROR_TYPES = (LLMRateLimitError, LLMTimeoutError) returns valid=True for any 429. The SDK maps every litellm.RateLimitError to LLMRateLimitError, including non-recoverable ones such as a LiteLLM proxy budget_exceeded or OpenAI insufficient_quota. A profile whose key has no budget left is reported as valid, and the user only finds out when conversations hang.
2. Slow. The 1-token ping runs through LLM's retry decorator with the profile's own policy (defaults num_retries=5, retry_min_wait=8, retry_max_wait=64), so validation blocks for ~2 minutes on any retryable error (429, connection error) before answering.

Seen in Agent Canvas (agent-server 1.48.0) with an openhands/ profile whose proxy key was over budget: saving/verifying a profile hung ~30–60 s and then passed.

Expected Behavior

• A non-recoverable 429 (budget / quota exhausted) returns valid=false with the redacted provider message.
• Validation answers quickly: the pre-flight call does not retry (or retries at most once with a short wait) and uses a short timeout, independent of the profile's runtime retry policy.
• Genuine, recoverable rate limits may still be non-blocking.

Actual Behavior

Repro against the real handler, with litellm stubbed to return the proxy's budget-exceeded 429:

uv run --python 3.13 --with openhands-agent-server==1.48.0 python repro_preflight_budget.py

Output:

response={'valid': True, 'error': None} llm_calls=5 elapsed=120s

Retries logged at +0 s, +8 s, +24 s, +56 s. With an unreachable base_url (no stub) the result is correctly valid=False, but it also takes 120 s.

Still present on main: profiles_router.py:276 (_TRANSIENT_ERROR_TYPES) and profiles_router.py:328-330 (llm.aresponses / llm.acompletion with the profile's retry settings).

Steps to Reproduce

1. Save the script below as repro_preflight_budget.py.
2. Run uv run --python 3.13 --with openhands-agent-server==1.48.0 python repro_preflight_budget.py.
3. Observe valid: True after ~120 s and 5 LLM calls.

Minimal Code Sample

import asyncio, time
from unittest.mock import patch

import litellm
from openhands.agent_server import profiles_router as pr
from openhands.sdk.llm import LLM

calls = 0

async def budget_exceeded(*args, **kwargs):
    global calls
    calls += 1
    raise litellm.RateLimitError(
        message='{"error":{"message":"Budget has been exceeded! Current cost: 1060.95, '
        'Max budget: 1016.82","type":"budget_exceeded","code":"429"}}',
        llm_provider="litellm_proxy",
        model="litellm_proxy/gpt-5.5",
    )

async def main():
    # Default retry policy (num_retries=5, retry_min_wait=8, retry_max_wait=64).
    llm = LLM(model="litellm_proxy/gpt-5.5", api_key="sk-test", base_url="http://proxy.invalid")
    body = pr.ValidateProfileRequest(llm=llm)
    with patch("openhands.sdk.llm.llm.litellm_acompletion", budget_exceeded), \
         patch("openhands.sdk.llm.llm.litellm_aresponses", budget_exceeded), \
         patch.object(pr, "get_cipher", lambda request: None):
        start = time.monotonic()
        resp = await pr.validate_profile(request=None, name="demo", body=body)
    print(f"response={resp.model_dump()} llm_calls={calls} elapsed={time.monotonic() - start:.0f}s")

asyncio.run(main())

Logs and Error Messages

litellm.RateLimitError: RateLimitError: Litellm_proxyException - {"error":{"message":"Budget has been exceeded! Current cost: 1060.956104022548, Max budget: 1016.82958895","type":"budget_exceeded","param":null,"code":"429"}}
INFO Profile 'gpt5.5' pre-flight hit a transient error (LLMRateLimitError); not blocking save.  profiles_router.py:333

Acceptance Criteria

• A 429 whose body indicates exhausted budget/quota (e.g. budget_exceeded, insufficient_quota) makes validate_profile return valid=false with a redacted error message.
• A plain, recoverable rate limit is still non-blocking (valid=true).
• The pre-flight call does not use the profile's runtime retry policy; the repro above returns in a few seconds with at most 2 LLM calls.
• Unit tests in tests/agent_server/ cover the budget-exceeded 429, a recoverable 429, and the no-long-retry behaviour.

Installation Method

pip / uv (openhands-agent-server via uvx, launched by Agent Canvas)

SDK Version

1.48.0 (also verified against main)

Version Confirmation

• I have confirmed this bug exists on the LATEST version of OpenHands SDK

Python Version

3.13

Model Name (if applicable)

openhands/gpt-5.5 via LiteLLM proxy

Operating System

macOS

Start in openhands-agent-server/openhands/agent_server/profiles_router.py at _TRANSIENT_ERROR_TYPES and validate_profile, then run the provided repro to observe the 429 handling and retry delay. Add focused coverage under tests/agent_server/ for exhausted quota, recoverable rate limits, and retry limits; done means the former returns invalid while recoverable errors remain non-blocking and validation completes quickly.