Upstream Issue Draft — secret-input parameter cache poisons later loop iterations . Loop/iteration nodes: `secret-input` tool parameter cache returns the first iteration's value for every iteration
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.
### Dify version
1.16.1
### Cloud or Self Hosted
Self Hosted (Docker)
### Steps to reproduce
# Upstream Issue Draft — secret-input parameter cache poisons later loop iterations
> Target repo: langgenius/dify
> Verified against the local source tree at /data/dify (DSL 0.7.0 era, `api/core/tools/utils/configuration.py`, `api/core/tools/tool_manager.py`, `api/core/tools/__base/tool.py`).
---
**Title:** Loop/iteration nodes: `secret-input` tool parameter cache returns the first iteration's value for every iteration
## Description
When a tool whose `form: form` parameter is `secret-input` (e.g. `langgenius/wecom`'s `wecom_group_bot` `hook_key`) is placed **inside a loop/iteration node** and its value is **dynamically bound** (a `mixed` template referencing the iteration item), every iteration after the first silently sends with the **first iteration's** value. In our case a daily-report workflow routed one WeCom group-bot message per regional group; all four messages landed in the first group because the first group's webhook key was frozen into a Redis cache keyed only by node.
### Root cause chain
1. `ToolManager.get_workflow_tool_runtime` (`api/core/tools/tool_manager.py:469-480`) builds `runtime_parameters` from the *current* variable pool (correct, per-iteration), then passes them through `ToolParameterConfigurationManager.decrypt_tool_parameters`.
2. `decrypt_tool_parameters` (`api/core/tools/utils/configuration.py`) reads/writes a Redis cache (`ToolParameterCache`, 86400 s TTL) keyed by `tool_parameter_secret:{tenant}:{provider}:{tool}:{identity_id}` where `identity_id = WORKFLOW.{app_id}.{node_id}` — **it does not include the iteration index or the input values**. On a cache hit it returns the *entire* cached parameter dict, discarding the freshly rendered values.
3. `Tool.invoke` (`api/core/tools/__base/tool.py:58-59`) then does `tool_parameters.update(self.runtime.runtime_parameters)`, so the stale cached secret **overwrites** the correct per-iteration value that the graph engine resolved.
Net effect: for secret-input parameters, iteration N>1 always uses iteration 1's value. Non-secret `form: llm` parameters (message `content`, `to_user`, …) are resolved per-iteration by the graph engine and are unaffected — which produces the confusing symptom of *distinct message contents all delivered to the same destination*.
## Steps to reproduce
1. Install the `langgenius/wecom` plugin (any recent version; `wecom_group_bot` declares `hook_key` as `secret-input`, `form: form`).
2. Create a workflow:
- a code node producing `messages: [{webhook_key: , content: "one"}, {webhook_key: , content: "two"}]` with two **different, both valid** group-bot webhook keys;
- a loop node over that array; inside it, a small code node surfacing `item.webhook_key` as `hook_key`, then a `wecom_group_bot` tool node with `hook_key` bound to `{{#.hook_key#}}` (mixed template) and `content` bound to the item content;
- run it.
3. Observe: both messages arrive in **group 1**. The node execution log shows the correct per-iteration `hook_key` in the node's *inputs*, confirming the graph engine resolved the right values and the corruption happens afterwards.
Minimal in-process repro against `ToolManager` (no UI needed):
```python
def get_hook(key_value):
vp = VariablePool.from_bootstrap(system_variables={}, user_inputs={})
vp.add(["pick", "hook_key"], key_value)
tool = ToolManager.get_workflow_tool_runtime(
tenant_id=TENANT, app_id="repro", node_id="group_send",
workflow_tool=spec_with(tool_configurations={
"hook_key": {"type": "mixed", "value": "{{#pick.hook_key#}}"},
"message_type": {"type": "constant", "value": "markdown_v2"},
}),
variable_pool=vp,
)
return tool.runtime.runtime_parameters["hook_key"]
get_hook("aaaaaaaa-0000-0000-0000-000000000001") # -> aaaaaaaa-...
get_hook("bbbbbbbb-0000-0000-0000-000000000002") # -> aaaaaaaa-... <-- stale cache hit
```
## Expected behavior
A cache entry for secret-input parameters must only be served when it was built from the **same encrypted inputs** as the current call. Dynamically bound secrets inside loops must resolve per iteration; statically configured secrets should keep benefiting from the decrypt cache.
## Suggested fix
Fingerprint the secret-form inputs and validate cache entries against it (keeps the decrypt-cache benefit for the static case, fixes the loop case, and treats legacy entries without a fingerprint as stale so deployments self-heal within the 24 h TTL):
```diff
--- a/api/core/tools/utils/configuration.py
+++ b/api/core/tools/utils/configuration.py
@@
import contextlib
+import hashlib
+import json
from copy import deepcopy
from typing import Any
@@
+_CACHE_FINGERPRINT_KEY = "__cache_fingerprint"
+
@@
class ToolParameterConfigurationManager:
@@
+ def _fingerprint(self, parameters: dict[str, Any]) -> str:
+ secret_parameters = self._merge_parameters()
+ material = {
+ parameter.name: parameters.get(parameter.name)
+ for parameter in secret_parameters
+ if parameter.form == ToolParameter.ToolParameterForm.FORM
+ and parameter.type == ToolParameter.ToolParameterType.SECRET_INPUT
+ }
+ canonical = json.dumps(material, sort_keys=True, default=str)
+ return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
+
def decrypt_tool_parameters(self, parameters: dict[str, Any]) -> dict[str, Any]:
@@
- cached_parameters = cache.get()
- if cached_parameters:
+ fingerprint = self._fingerprint(parameters)
+ cached_parameters = cache.get()
+ if (
+ cached_parameters
+ and isinstance(cached_parameters.get(_CACHE_FINGERPRINT_KEY), str)
+ and cached_parameters[_CACHE_FINGERPRINT_KEY] == fingerprint
+ ):
+ cached_parameters.pop(_CACHE_FINGERPRINT_KEY, None)
return cached_parameters
@@
if has_secret_input:
- cache.set(parameters)
+ cache.set({**parameters, _CACHE_FINGERPRINT_KEY: fingerprint})
```
The fingerprint intentionally covers **only secret-form parameters**: non-secret parameters (e.g. message content) legitimately change between iterations without invalidating cached secret decrypts. Only the secret inputs gate cache validity.
## Environment
- Self-hosted from source, current `main` (also present in the 1.16 line per source inspection)
- Plugins: `langgenius/wecom 0.0.10` (any tool with a `secret-input` `form: form` parameter is affected)
- Loop node, sequential mode (`error_handle_mode: continue-on-error`)
---
### ✔️ Expected Behavior
A cache entry for secret-input parameters must only be served when it was built from the same encrypted inputs as the current call. Dynamically bound secrets inside loops must resolve per iteration; statically configured secrets should keep benefiting from the decrypt cache.
### ❌ Actual Behavior
_No response_
Contributor guide
Research direction
Start with api/core/tools/utils/configuration.py, then trace the parameter flow through api/core/tools/tool_manager.py and api/core/tools/__base/tool.py. Run the minimal in-process ToolManager reproduction with two different hook_key values and inspect the cache behavior. Done means each loop iteration preserves its resolved secret-input value while statically configured secrets still benefit from caching.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100