[Refactor/Chore] Batch-load dataset document list response details
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for refactors or chores; if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.
### Description
The console dataset document list endpoint resolves several response fields—hit counts, data-source details, process rules, and metadata—through per-document database lookups. With a full page, the number of queries grows linearly with the page size. When `fetch=true`, completed and total segment counts add two more queries per document.
### Motivation
Batch the related rows for all documents on the current page and reuse them during response serialization so the query count remains bounded. Preserve the existing response shape and counting semantics, including re-segment handling and tenant/dataset ownership boundaries.
### Additional Context
Target endpoint: `GET /console/api/datasets/{dataset_id}/documents`.
Add regression coverage that compares the SELECT count for one document and multiple documents, along with response-value checks.
Contributor guide
Research direction
Start by tracing GET /console/api/datasets/{dataset_id}/documents and identify where response fields and fetch=true segment counts trigger per-document lookups. Add regression coverage comparing SELECT counts for one versus multiple documents, while checking response values, re-segment handling, and tenant/dataset ownership boundaries. Done means the response shape and counting semantics are preserved with bounded query growth.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend-api-design, databases
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100