langgenius / langgenius/dify

[Bug] #context_files# is silently empty for published-app end users on 1.16.1 (works in Studio preview) — retrieved knowledge attachments never reach the LLM

Open
#42,277 6 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the Contributing Guide and Language Policy.
- [x] This is only for bug report, if you have a question, please head to Discussions.
- [x] I have searched for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭。
- [x] Please do not modify this template and fill in all the required fields.

### Dify version

`1.16.1` (Self Hosted / Docker). Image built from the 1.16.1 release plus a local Weaviate UUID patch (unrelated area).

**PR #36175 (`fix(api): allow LLM nodes to access retrieved knowledge files`, merged 2026-05-14, shipped in v1.14.2) is present in this build.** I verified both of its artifacts are in the running container:

- `core/app/file_access/scope.py` — the `granted_upload_file_ids` "execution-local" scope fields
- `core/workflow/node_factory.py` — `_build_retriever_segment_access_checker`

### Summary

For a **published** Advanced Chat app invoked through the **service API** as an **end user**, the LLM node never receives retrieved knowledge-base image attachments — `#context_files#` is not merely empty, the key is **absent** from `workflow_node_executions.inputs`. The same workflow, same dataset, same model and same vision configuration, invoked through **Studio preview**, does receive them.

There is **no error and no log line** — `#context#` is populated normally, the run succeeds, and the answer is produced from the caption text. This is a *silent* failure.

### This is likely NOT issue #36174

#36174 (fixed by #36175) has the same trigger — published app + end user — but fails **loudly**:

```
Node llm failed with ABORT strategy: Invalid upload file
RuntimeError: Invalid upload file
```

We do **not** get that error. In our build the loader path passes the ownership check without raising, and returns an empty list instead. So the #36175 fix appears to be in place, but something upstream of it still refuses the attachment for end users.

### Steps to reproduce

1. Knowledge base with per-segment image attachments (`segment_attachment_bindings` + `upload_files` are fully populated; attachment preview URLs return `200 image/jpeg`).
2. Advanced Chat app: `Knowledge Retrieval` node → `LLM` node. LLM node: **Context** enabled and bound to the retrieval node's `result`; **Vision** enabled with the file variable set; model has the `vision` feature (`deepseek-flash` / `deepseek-v4-flash-vision-exp`).
3. Run it in **Studio preview** as the workspace account → LLM node input contains `#context_files#` with **5** files.
4. **Publish**, then invoke the same app through `POST /v1/chat-messages` with the same question → LLM node input has **no `#context_files#` key**, 0 files.

### Evidence

Same app, **draft graph and published graph are field-by-field identical** (52 nodes; same context selector; same vision config; same model; same dataset on both retrieval nodes), so the only variable is the invoking identity:

| run | invocation | `user_from` | `#context#` length | `#context_files#` |
|---|---|---|---|---|
| A | service API `/v1/chat-messages` | `END_USER` | 6243 chars | **key absent** |
| B | service API `/v1/chat-messages` | `END_USER` | 10323 chars | **key absent** |
| C | service API `/v1/chat-messages` | `END_USER` | 4516 chars | **key absent** |
| D | **Studio preview** | `ACCOUNT` | 4505 chars | **5 files** ✅ |

Query used (authoritative — the SSE `node_started` event carries a *pre-context-collection* snapshot and must not be used for this):

```sql
SELECT created_at,
(inputs::jsonb ? '#context_files#') AS has_context_files,
jsonb_array_length(inputs::jsonb -> '#context_files#') AS n_files,
length(inputs::jsonb ->> '#context#') AS context_len
FROM workflow_node_executions
WHERE workflow_run_id = '' AND title = '';
```

The retrieved segment is not the problem — it has attachments and they resolve:

```
segment_attachment_bindings for that segment_id : 3
of which JOIN upload_files : 3
```

### Environment / configuration that I believe is correct

- KB: high_quality, `is_multimodal = false`, text embedding `bge-m3` (attachments are produced by our own ingestion pipeline and written into segment content as markdown image references, which Dify converts into `upload_files` + `segment_attachment_bindings`).
- LLM node: `context.enabled = true`, `context.variable_selector` = the retrieval node's `result`; `vision.enabled = true`, `vision.configs.variable_selector = ["sys","files"]`, `detail = high`; model advertises `vision` in its plugin model schema.
- I also tested with a **direct** Knowledge Retrieval node added next to the LLM node (not inside an iteration) and rebound the context to it — same result (0 files over API).

### What I have already ruled out (to save you time)

| Hypothesis | Result |
|---|---|
| Iteration `is_parallel = true` (thread isolation) | ❌ Turning parallel mode **off** changes nothing |
| Retrieval inside an iteration sub-graph | ❌ A direct, non-iteration retrieval node reproduces it |
| Flat list → wrong Segment class | ❌ `build_segment([{...}])` and `build_segment_with_type(ARRAY_OBJECT, [...])` both yield `ArrayObjectSegment`, so the `isinstance(..., ArrayObjectSegment)` check in `_build_retriever_segment_access_checker` can pass |
| Iteration output loses `metadata` | ❌ DB shows the flattened output items carry `metadata._source = "knowledge"` and `metadata.segment_id` |
| A downstream Code node | ❌ The context is bound to the retrieval/iteration output, not to the Code node |
| `copy_context` / extra thread pool in graphon | ❌ `copy_context` does not appear anywhere in the `graphon` package, and `ThreadPoolExecutor` appears only in the iteration node — so retrieval and the LLM node appear to execute in the same context |

### Reading of the code, and where I get stuck

`DifyRetrieverAttachmentLoader.load()` (`core/workflow/node_runtime.py`) has two guards:

```python
if not is_retriever_segment_access_granted(segment_id):
return []
if self._segment_access_checker is not None and not self._segment_access_checker(segment_id):
return []
```

`is_retriever_segment_access_granted` (`core/app/file_access/scope.py`) returns `True` when `scope is None` or `not scope.requires_user_ownership` — i.e. it only actually checks the set for `END_USER`.

Since run **D** (ACCOUNT) successfully loads 5 files through the **second** guard, the second guard is satisfied by this graph. That implies the **first** guard is what returns `False` for `END_USER`, i.e. `granted_retriever_segment_ids` is empty at read time, even though the retrieval path calls `grant_retriever_segment_access([str(segment.id)])` (`core/rag/retrieval/dataset_retrieval.py`, in `knowledge_retrieval()`, in the same block that sets `retriever_from="workflow"` — and the stored retrieval output does show `retriever_from: "workflow"`, so that block definitely executed).

I could not reconcile "grant is written" with "grant is not visible", given there is no thread/context boundary that I can find in `graphon`. If you can tell me what else could separate the grant write from the read (or what to log to confirm it), I am happy to run that instrumentation on my instance and report back.

### Expected Behavior

For a published app invoked by an end user, retrieved knowledge attachments should resolve for the LLM node — as they do for the workspace account in Studio preview — while tenant isolation and the "segment must be present in the selected retrieval context" constraint are preserved.

### Actual Behavior

`#context_files#` is absent for end-user invocations, with no error and no log output. Vision-capable models therefore never see the retrieved images; only the text (including our caption text) reaches them.

Contributor guide

Open the contributing guide

Research direction

Trace the END_USER access scope from grant_retriever_segment_access in core/rag/retrieval/dataset_retrieval.py through core/app/file_access/scope.py to DifyRetrieverAttachmentLoader.load in core/workflow/node_runtime.py; compare it with the ACCOUNT path and inspect the related checker in core/workflow/node_factory.py. Add focused instrumentation or tests around the grant and read points to identify why the segment ID is not visible. Done means published API runs resolve retrieved attachment files without weakening ownership or retrieval-context checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.