google / google/adk-js

[QUESTION] How can we implement exponentialBackoff in the ADK (reusing or not the retry mechanism in `js-genai`)?

Open
#259 3 comments 1 reaction 1 assignee Claimed by @ScottMansfield View on GitHub
enhancement question status/in-progress
Dominant language
TypeScript
Stars
1.4k
Forks
205
Avg merge
3d 16h
Merged PRs (30d)
92

Description

Hello,

When we use [js-genai](https://github.com/googleapis/js-genai/tree/b6c5d189bc0ae98550c45333c5e7a9bc43648dd3) directly, outside ADK, we can apply [httpOptions](https://github.com/googleapis/js-genai/blob/b6c5d189bc0ae98550c45333c5e7a9bc43648dd3/src/types.ts#L2020) to [modify](https://github.com/googleapis/js-genai/blob/b6c5d189bc0ae98550c45333c5e7a9bc43648dd3/src/types.ts#L1996C1-L2000C2) the default retry attempts. Since this is often not enough and one cannot modify the [default retry codes](https://github.com/googleapis/js-genai/blob/b6c5d189bc0ae98550c45333c5e7a9bc43648dd3/src/_api_client.ts#L48C7-L48C38), in our apps on production we wrap e.g., `chatSession.chat.sendMessage` with our own exponentialBackoff functionality. It covers other cases, like and error which just prints `fetch failed` too.

Now, we would like to apply the same, but how can we do this in the ADK? From what we see, ADK abstracts away the direct SDK calls, so we cannot easily wrap individual SDK invocations, can we?

One option was to extend Gemini's [*generateContentAsync](https://github.com/google/adk-js/blob/e8b2caeb219de7d84e1a9e399a52fe19cb9c70c9/core/src/models/google_llm.ts#L132C18-L132C39) method, but that seems like too much, doesn't it?

Note:
For the record, it seems Python guys allowed to [modify](https://github.com/google/adk-python/issues/1214#issuecomment-3144496751) the http retry options, but I'm not sure about implementing a custom exponential backoff logic.

Thanks,
Iliyan

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.