kestra-io / kestra-io/plugin-ai

Fall back to another model provider when the primary one fails

Open
#418 2 comments 0 reactions 0 assignees View on GitHub
area/plugin good first issue
Dominant language
Java
Stars
9
Forks
24
Avg merge
1d 23h
Merged PRs (30d)
15

Description

### Feature description

### Why

LLM calls fail for reasons that have nothing to do with the prompt. Rate limits, 5xx from the provider, timeouts. This is the normal failure mode in production, not the rare one.

Today a task points at exactly one model provider. retry re-runs the same task against the same provider, so it does not help when the provider itself is the problem. The only way to fail over right now is to wrap the task in a Sequential and put a second copy of it in errors. That means duplicating the prompt, the tools, the guardrails and the memory config for every fallback level, and downstream tasks end up with two possible places to read the output from.

This came up in a Kestra Cloud 2.0 evaluation, where the reviewer built the pattern by hand and flagged it as one of the gaps for AI-heavy pipelines.

### What

Let a task declare an ordered list of model providers instead of one. The first one runs. If it fails with a provider-side error, we move to the next and keep the same prompt, tools, memory and guardrails. Downstream tasks read the result from one place, whatever provider answered, and the logs say which providers were tried and why each was dropped.

Failover should only cover provider-side failures. A guardrail violation, a bad prompt or a schema the model cannot satisfy should still fail the task. Retrying those somewhere else just spends the money twice.

### Acceptance criteria

- A task can declare more than one model provider, in order.
- On a provider-side failure the next provider runs with the same prompt, tools, memory and guardrails.
- Downstream tasks read the output from one place regardless of which provider answered.
- The execution logs name every provider tried and the reason each one was dropped.
- Guardrail violations and prompt or schema errors do not trigger failover.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files or tests are named. Start by tracing how a task currently configures one model provider, handles provider errors and retries, and exposes its result to downstream tasks. Define provider-side failures separately from guardrail, prompt and schema errors, then verify the acceptance criteria through focused execution and logging tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.