fix(tasks): rollback poisoned session in async_workflow failure path
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 20h 50m
- Merged PRs (30d)
- 586
Description
## async_workflow_tasks rollback-poisoning the failure-bookkeeping session
`api/tasks/async_workflow_tasks.py` lines 188-200 — the `except` handler for the workflow generation call tries to update `trigger_log` and commit on the same session that the generator ran on. If the generator raises a SQLAlchemy flush error (any `IntegrityError`, `DataError`, `OperationalError`, etc.), the session is left in a failed-transaction state, and the subsequent `session.commit()` raises `PendingRollbackError` instead of recording the failure.
### Trigger scenario
Any async workflow execution whose database operations inside `generator.generate(...)` raise a flush-time SQLAlchemy error. The Celery task then exits with a second DB exception, the trigger log is never updated, and the workflow remains stuck in `RUNNING` state with no error recorded.
### Steps to reproduce
The bug is non-deterministic — it requires any flush-time DB error during workflow execution. Common causes:
- A `WorkflowTriggerLog` insert that hits a unique-constraint conflict.
- A message insert that violates a NOT NULL or check constraint.
- Any DML whose prepared statement is rejected by the server (e.g. data type mismatch).
When any of these happen, the failure-path code that is supposed to mark the trigger as `FAILED` is itself swallowed by `PendingRollbackError`.
### Expected
`trigger_log.status = WorkflowTriggerStatus.FAILED` is committed, `trigger_log.error` contains the original exception, monitoring can see the failure.
### Actual
`session.commit()` at line 200 raises `PendingRollbackError`. The Celery task exits with that exception. The trigger log remains `RUNNING` (or whatever the previous status was) and the original error message is lost.
### Suggested fix
Add `session.rollback()` at the start of the except block, then reload `trigger_log` from the repository so the in-memory instance is bound to the now-clean session, then update and commit as before. Roughly:
```python
except Exception as e:
session.rollback()
trigger_log = trigger_log_repo.get_by_id(trigger_log.id)
...
session.commit()
```
This guarantees the bookkeeping transaction succeeds independently of whatever failure left the session in a bad state.
Contributor guide
Research direction
Start in api/tasks/async_workflow_tasks.py around lines 188-200 and trace the exception path after generator.generate(...). Reproduce or inspect a flush-time SQLAlchemy failure, then verify that the trigger log records the original error with status FAILED and no PendingRollbackError replaces it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlalchemy
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100