langgenius / langgenius/dify

Batch segment import stays stuck at waiting when CSV processing fails

Open Beginner friendly
#38,862 1 comment 1 reaction 0 assignees View on GitHub
project#dify
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Describe the bug

When importing segments from a CSV, only failures during the initial setup phase mark the
job as failed. If the CSV parsing or the indexing work fails afterwards, the Redis job
status is left at `waiting` (which is written without a TTL), so the status endpoint keeps
returning `waiting` and the frontend keeps polling indefinitely. The user sees an import
that never finishes and never reports an error.

### Steps to reproduce

1. Create a knowledge base document and open "Batch add segments".
2. Upload a CSV that fails during processing, for example:
- an empty CSV (header only, no rows), or
- a QA dataset with a CSV that has only one column (no answer column).
3. Watch the batch import status.

### Expected behavior

The job status becomes `error`, and the UI stops polling and shows the failure.

### Actual behavior

The job status stays at `waiting` forever. The worker logs a traceback, but the UI never
learns about it and keeps polling every few seconds.

### Root cause

In `api/tasks/batch_create_segment_to_index_task.py`, only the setup `session` block is
wrapped in a try/except that sets the Redis status to `error`. The CSV parsing and the
segment/vector creation that follow are not guarded, so an exception there skips the
`completed`/`error` update entirely. The initial `waiting` value is written with `setnx`
(no TTL) in `api/controllers/console/datasets/datasets_segments.py`, so it never expires.

### Environment

- Dify version: main (self-hosted / from source)

I'd like to work on this and will open a PR.

Contributor guide

Open the contributing guide

Research direction

Start in api/tasks/batch_create_segment_to_index_task.py and trace the CSV parsing and segment/vector creation after the setup session block. Reproduce with an empty CSV or one-column QA CSV, then verify that failures set the Redis status to error and the status endpoint no longer leaves the UI polling indefinitely; inspect api/controllers/console/datasets/datasets_segments.py to understand the initial waiting value.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
api, backend, databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.