[Feature] Retry failed document indexing through the Knowledge Service API
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the Contributing Guide and Language Policy.
- [x] I have searched for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report.
- [x] Please do not modify this template and fill in all the required fields.
### 1. Is this request related to a challenge you're experiencing? Tell me about your story.
We use the Knowledge Service API to upload TXT documents and poll their asynchronous indexing status. If indexing fails, our automation needs to retry the existing document after the underlying problem is resolved.
The Console supports this through POST /console/api/datasets/{dataset_id}/retry with a document_ids array. However, this endpoint requires a logged-in Console account, while our integration uses a dataset API key. We could not find an equivalent public Service API endpoint in the current main branch.
Depending on browser login cookies and CSRF tokens makes unattended team automation fragile. Deleting and recreating the document changes its ID and requires rebuilding local mappings. Updating a document by file is also not the same operation as explicitly retrying failed indexing with the existing stored source.
Please expose a supported Service API operation to retry failed document indexing using dataset API key authentication, without requiring a Console session or re-uploading the source file.
### 2. Additional context or comments
Suggested contract (subject to maintainer feedback):
POST /v1/datasets/{dataset_id}/documents/retry
Authorization: Bearer
Content-Type: application/json
{"document_ids": ["document-uuid"]}
The endpoint should reuse existing indexing retry logic, preserve document IDs, validate dataset/tenant access and document states, apply existing rate limits, and return an asynchronous acknowledgement. Clients would continue polling indexing status; acknowledgement would not mean indexing has completed.
Relevant code:
- api/controllers/console/datasets/datasets_document.py: DocumentRetryApi
- api/services/dataset_service.py: DocumentService.retry_document
- api/tasks/retry_document_indexing_task.py
- api/controllers/service_api/dataset/document.py
This request is specifically for API access to explicit retries, not unlimited automatic retries or bypassing authentication. Source inspected: main at 3e44b51d7a.
### 3. Can you help us with this feature?
- [x] I am interested in contributing to this feature.
Contributor guide
Research direction
Start by reading api/controllers/console/datasets/datasets_document.py, api/services/dataset_service.py, api/tasks/retry_document_indexing_task.py, and api/controllers/service_api/dataset/document.py to trace the existing retry flow and service authentication. Done means a documented service endpoint accepts dataset API-key requests, preserves document IDs, validates access and states, applies existing rate limits, and returns an asynchronous acknowledgement while clients continue polling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, authentication, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100