Multimodal rerank maps results to the wrong document after skipping a missing image upload
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for a bug report. Questions belong in [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I searched existing issues, including closed ones.
- [x] I am submitting this report in English.
- [x] I did not modify the required fields of the bug report template.
### Dify version
Current `main` at `09054169c42100b8ff5ebc1163636220fa7162bc`.
### Cloud or Self Hosted
Self Hosted (Source)
### Steps to reproduce
1. Configure a multimodal dataset with a vision-capable reranking model.
2. Leave an image vector candidate whose `UploadFile` row no longer exists. This state is production-reachable when vector/keyword cleanup in `clean_document_task` fails: the task explicitly continues PostgreSQL and storage cleanup while leaving vector orphans for later reaping.
3. Run image-query semantic retrieval with the missing image candidate followed by a valid text candidate.
4. Have the reranking provider return result index `0` for the only document it received.
A minimal mocked reproduction on the commit above produces:
```text
model_input_count=1
model_input_content=valid text
returned_doc_id=missing-image
```
`RerankModelRunner.fetch_multimodal_rerank()` omits the missing image from `docs`, but still appends its source `Document` to `unique_documents`. `RerankModelRunner.run()` later resolves the provider's result index against that longer, shifted list.
### ✔️ Expected Behavior
A candidate that cannot be included in the model input should also be omitted from the parallel source-document list. Result index `0` should therefore map to the valid text document that was actually sent to the model.
### ❌ Actual Behavior
The missing image remains at index `0` in `unique_documents`, so the valid text result receives the missing image's metadata and provider. Later candidates are similarly shifted.
### Additional context
The existing multimodal reranker test named `test_fetch_multimodal_rerank_skips_missing_image_upload` uses a text-query mock and accepts the stale source-document list, so it does not cover the image-query result mapping.
I can submit a minimal fix and focused regression test if the maintainers agree with this direction.
AI assistance was used to inspect the code paths, build the isolated reproduction, and draft this report. I reviewed and verified the technical claims against the referenced commit.
Contributor guide
Research direction
Start at RerankModelRunner.fetch_multimodal_rerank() and RerankModelRunner.run(), then read the existing test named test_fetch_multimodal_rerank_skips_missing_image. Add an image-query regression case using a missing image followed by valid text, and run the multimodal reranker tests; done means provider result indexes map to the documents actually sent to the model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- ai, backend, databases, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100