langgenius / langgenius/dify

bug(agent): CreateRunRequest.idempotency_key is accepted but not enforced

Open
#39,685 1 comment 2 reactions 0 assignees View on GitHub
🐞 bug
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.

### Dify version

main branch (base commit 59fb603)

### Cloud or Self Hosted

Self Hosted (Source)

### Steps to reproduce

1. Start the standalone `dify-agent` run server with Redis.
2. Send a `POST /runs` request containing an `idempotency_key`.
3. Replay the exact same request with the same idempotency key.
4. Observe that the server creates another run id and starts another background task.

The same behavior occurs when concurrent requests reach different server
processes sharing the same Redis prefix.

## Actual behavior

`RunScheduler.create_run()` always creates a new UUID, persists a new run record,
and starts a new task. `CreateRunRequest.idempotency_key` is accepted but is not
used during run admission.

## Expected behavior

During the configured run retention window:

- an exact replay should return the original run id and current status;
- no second background task should be started;
- reusing the key with a different normalized request should return HTTP 409;
- requests without a key should retain the existing behavior.

## Impact

Retries can cause duplicate model invocations, repeated tool side effects,
additional cost, and inconsistent workflow state. Agent v2 workflow, retry, and
cleanup callers already generate idempotency keys and expect the backend to
deduplicate requests.

## Proposed solution

Atomically associate a hashed idempotency key and normalized request fingerprint
with the original run in Redis. Only digests and the run id should be persisted,
without storing model credentials or the original request.

I would like to work on this issue. I have a focused implementation with
scheduler, Redis storage, HTTP route, documentation, and regression tests ready.
Could a maintainer please assign this issue to me?

### ✔️ Expected Behavior

During the configured run retention window:

- An exact replay with the same idempotency key and normalized request should
return the original run id and current status.
- The replay should not start another background task.
- Reusing the key with a different normalized request should return HTTP 409.
- Requests without an idempotency key should continue creating new runs.

### ❌ Actual Behavior

`CreateRunRequest.idempotency_key` is accepted but ignored during run admission.

`RunScheduler.create_run()` always generates a new run id, persists another run
record, and starts another background task. Therefore, replaying the same request
can repeat model invocations and tool side effects, increase cost, and produce
inconsistent workflow state.

Agent v2 workflow, retry, and cleanup callers already generate idempotency keys
and expect the backend to deduplicate these requests.

I would like to work on this issue and already have a focused implementation
with regression tests ready. Could a maintainer please assign it to me?

Contributor guide

Open the contributing guide

Research direction

Start at the POST /runs route, CreateRunRequest, and RunScheduler.create_run(); trace how Redis is used by the standalone dify-agent run server. Verify exact keyed replays return the original run id and status without starting a task, conflicting normalized requests return HTTP 409, and requests without keys keep existing behavior. Regression tests should cover concurrent requests across processes and retention cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
api, backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.