langgenius / langgenius/dify

Changing the embedding model can leave completed summaries missing from the active vector index

Open
#42,234 0 comments 1 reaction 0 assignees View on GitHub
🐞 bug
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.

### Dify version

main @ 79effdd498a0c53218fe85c11f565b680468e455 (api/pyproject.toml: 1.17.1)

### Cloud or Self Hosted

Self Hosted (Source)

### Steps to reproduce

Changing the embedding model can leave existing summaries marked `completed` even though their vectors are missing from the active collection. There are two affected paths: independent document/summary rebuild tasks from knowledge base settings, and missing summary re-vectorization when publishing a knowledge pipeline.

Prerequisites: Qdrant, High Quality indexing, General chunking, an enabled summary index, and a document with a successfully generated and retrievable summary.

**Knowledge base settings**

1. Change the knowledge base's embedding model from A to B.
2. Let the summary re-vectorization task finish before the document reindex task clears the dataset's vectors. The two tasks are dispatched independently to `dataset_summary` and `dataset`, so this is a possible execution order.
3. Wait for document reindexing to finish.
4. Check the summary's SQL status and look up its vector in the active model B collection.

For a deterministic task-level reproduction, start from a dataset with an existing completed summary, commit model B and its collection binding, and then run:

```python
from tasks.deal_dataset_vector_index_task import deal_dataset_vector_index_task
from tasks.regenerate_summary_index_task import regenerate_summary_index_task

regenerate_summary_index_task.run(dataset_id, "embedding_model_changed", True)
deal_dataset_vector_index_task.run(dataset_id, "update")
```

The summary exists in B's collection immediately after the first call, but disappears after the second. Reversing the calls is the passing control.

**Knowledge pipeline publishing**

1. Start with an already published pipeline whose knowledge base has completed summaries.
2. Change the embedding model in the knowledge configuration, keep the chunking/indexing settings unchanged, and publish again.
3. After document reindexing, inspect summary vectors in the active model B collection. Existing summaries have not been migrated there.

I reproduced the task paths using the actual Dify tasks, `Vector` factory and Qdrant adapter with the official `QdrantClient(location=":memory:")` engine and SQLite. Embeddings were deterministic; Redis and broker calls were replaced at their boundaries. The settings-save and pipeline-publish service entry points were also executed separately to verify the tasks they dispatch. This is a source-level reproduction, not a full browser/HTTP/Celery-cluster end-to-end test. Other vector backends and chunking modes were not runtime-tested.

### ✔️ Expected Behavior

Once the model switch and its rebuild tasks finish, existing summaries should be vectorized with the new model and remain retrievable from the active collection. SQL `completed` records should agree with the stored index.

### ❌ Actual Behavior

The document vectors are rebuilt successfully, but the summaries remain `completed` in SQL while their vectors are absent from the active collection.

| Execution path | Document vector | Summary SQL status | Active summary vector | Summary search hits |
| --- | --- | --- | --- | --- |
| Summary rebuild → document rebuild | Present | completed | Missing | 0 |
| Document rebuild → summary rebuild (control) | Present | completed | Present | 1 |
| Pipeline model update | Present | completed | Missing | 0 |

The regression assertions produced `2 failed, 1 passed`; both failures were `completed summary vector disappeared`. Other datasets' vectors in the same collection were preserved.

**Cause**

- [The settings service dispatches both tasks independently](https://github.com/langgenius/dify/blob/79effdd498a0c53218fe85c11f565b680468e455/api/services/dataset_service.py#L836-L845). Document reindexing subsequently calls `clean(dataset, None)`, which deletes the dataset's summary vectors as well as its document vectors. The subsequent document load does not rebuild summaries.
- [Qdrant deletion filters by the dataset's group_id](https://github.com/langgenius/dify/blob/79effdd498a0c53218fe85c11f565b680468e455/api/providers/vdb/vdb-qdrant/src/dify_vdb_qdrant/qdrant_vector.py#L285-L302), without distinguishing summary metadata. This affects the target dataset, not every dataset sharing the collection.
- [Pipeline settings dispatch only the document update task](https://github.com/langgenius/dify/blob/79effdd498a0c53218fe85c11f565b680468e455/api/services/dataset_service.py#L1344-L1351). With Qdrant's model-specific collection binding, existing summary vectors can remain in the old collection without being copied to the new one.

Related: [#40961](https://github.com/langgenius/dify/issues/40961) / [#40965](https://github.com/langgenius/dify/pull/40965) concern tasks reading the previous model before commit. This reproduction starts both tasks after the new model and binding are committed, so the missing-summary problem is independent of that transaction timing.

**Proposed fix**

Have both document update tasks enqueue summary re-vectorization after their cleanup, document loading and SQL commits finish, preserving the `dataset_summary` queue. Remove the independent summary dispatch from the settings service.

A local patch using this ordering makes all three vector-store cases pass. It also covers partial document failures, no completed documents and existing summaries with the summary setting disabled. I would like to contribute this fix with regression tests.

Contributor guide

Open the contributing guide

Research direction

Start with api/services/dataset_service.py and the task entry points tasks/deal_dataset_vector_index_task.py and tasks/regenerate_summary_index_task.py; reproduce the ordering where summary re-vectorization is followed by document cleanup. Read the Qdrant adapter at api/providers/vdb/vdb-qdrant/src/dify_vdb_qdrant/qdrant_vector.py, then add regression coverage for settings and pipeline model changes. Done means completed summaries remain present and searchable in the active model collection after document reindexing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.