langgenius / langgenius/dify

[Feature] Retry failed document indexing through the Knowledge Service API

Open
#42,104 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks
- [x] I have read the Contributing Guide and Language Policy.
- [x] I have searched for existing issues, including closed ones.
- [x] I confirm that I am using English to submit this report.
- [x] Please do not modify this template and fill in all the required fields.

### 1. Is this request related to a challenge you're experiencing? Tell me about your story.
We use the Knowledge Service API to upload TXT documents and poll their asynchronous indexing status. If indexing fails, our automation needs to retry the existing document after the underlying problem is resolved.

The Console supports this through POST /console/api/datasets/{dataset_id}/retry with a document_ids array. However, this endpoint requires a logged-in Console account, while our integration uses a dataset API key. We could not find an equivalent public Service API endpoint in the current main branch.

Depending on browser login cookies and CSRF tokens makes unattended team automation fragile. Deleting and recreating the document changes its ID and requires rebuilding local mappings. Updating a document by file is also not the same operation as explicitly retrying failed indexing with the existing stored source.

Please expose a supported Service API operation to retry failed document indexing using dataset API key authentication, without requiring a Console session or re-uploading the source file.

### 2. Additional context or comments
Suggested contract (subject to maintainer feedback):

POST /v1/datasets/{dataset_id}/documents/retry
Authorization: Bearer
Content-Type: application/json

{"document_ids": ["document-uuid"]}

The endpoint should reuse existing indexing retry logic, preserve document IDs, validate dataset/tenant access and document states, apply existing rate limits, and return an asynchronous acknowledgement. Clients would continue polling indexing status; acknowledgement would not mean indexing has completed.

Relevant code:
- api/controllers/console/datasets/datasets_document.py: DocumentRetryApi
- api/services/dataset_service.py: DocumentService.retry_document
- api/tasks/retry_document_indexing_task.py
- api/controllers/service_api/dataset/document.py

This request is specifically for API access to explicit retries, not unlimited automatic retries or bypassing authentication. Source inspected: main at 3e44b51d7a.

### 3. Can you help us with this feature?
- [x] I am interested in contributing to this feature.

Contributor guide

Open the contributing guide

Research direction

Start by reading api/controllers/console/datasets/datasets_document.py, api/services/dataset_service.py, api/tasks/retry_document_indexing_task.py, and api/controllers/service_api/dataset/document.py to trace the existing retry flow and service authentication. Done means a documented service endpoint accepts dataset API-key requests, preserves document IDs, validates access and states, applies existing rate limits, and returns an asynchronous acknowledgement while clients continue polling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, authentication, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.