langgenius / langgenius/dify

Hit testing fails with "Can't operate on closed transaction inside context manager"

Open Beginner friendly
#38,998 5 comments 1 reaction 0 assignees View on GitHub
🐞 bug 1.16.0 project#dify
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.

### Dify version

1.16.0-rc1

### Cloud or Self Hosted

Self Hosted (Docker)

### Steps to reproduce

1. Deploy Dify `1.16.0-rc1` locally using Docker Compose.
2. Create a knowledge base.
3. Upload a document and wait for indexing to complete.
4. Open the knowledge base retrieval test page.
5. Enter a query that matches one or more indexed segments.
6. Run the retrieval test.

The API returns HTTP 500. The relevant API log is:

```text
sqlalchemy.exc.InvalidRequestError: Can't operate on closed transaction inside context manager.
Please complete the context manager before emitting further commands.

File "/app/api/services/hit_testing_service.py", line 188, in retrieve
return cls.compact_retrieve_response(query, all_documents, session=session)

File "/app/api/services/hit_testing_service.py", line 244, in compact_retrieve_response
"records": cls._dump_retrieval_records(session, records)

File "/app/api/services/hit_testing_service.py", line 72, in _dump_retrieval_records
for document in session.scalars(
select(DatasetDocument).where(DatasetDocument.id.in_(document_ids))
).all()
```

The issue is reproducible when the retrieval returns matching document segments.

### ✔️ Expected Behavior

The retrieval test should complete successfully and display the matched document segments without returning an internal server error.

### ❌ Actual Behavior

The vector retrieval returns matching segments, but the API responds with HTTP 500 when it attempts to load the related document information.

`HitTestingService.retrieve()` calls `session.commit()` and then calls `compact_retrieve_response()`, which executes another query using the same session. Because the controller manages the session inside a transaction context, the explicit commit closes the transaction before the document query is executed.

Changing the internal `session.commit()` to `session.flush()` allows the outer transaction context to manage the final commit and prevents the error.

Contributor guide

Open the contributing guide

Research direction

Start in api/services/hit_testing_service.py, especially retrieve(), compact_retrieve_response(), and _dump_retrieval_records(), then inspect how the controller manages the SQLAlchemy session transaction. Reproduce with the Docker Compose retrieval test using a query that matches indexed segments. Done means the test returns matched document segments without the closed-transaction HTTP 500.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlalchemy
Domain
api, backend, databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.