langgenius / langgenius/dify

V2 archive delete fails when historical Parquet lacks a later-added nullable column

Open
#41,354 0 comments 1 reaction 1 assignee Claimed by @zhaohao1004 View on GitHub
🐞 bug cloud
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Self Checks

- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.

### Dify version

Source-verified on `main` at `3765a41b8c9747ef881ac7ef59b36fb3755932d1`. The issue applies to deployments containing #39001 and the later nullable-column migration.

### Cloud or Self Hosted

Cloud

### Steps to reproduce

After [#39001](https://github.com/langgenius/dify/pull/39001), V2 workflow archive deletion validates that the remaining live rows are an unchanged subset of the archived Parquet rows before deleting anything.

This fails for archive bundles written before a nullable column was added to `workflow_node_executions`.

1. Use a historical V2 archive bundle whose `workflow_node_executions.parquet` was written before `agent_workspace_binding_id` existed.
2. Keep the corresponding live `workflow_node_executions` row after the migration, with `agent_workspace_binding_id = NULL`.
3. Verify that live and archive row counts and row ID sets match, and that all shared fields have identical values.
4. Run:

```shell
flask delete-archived-workflow-runs --target-month --dry-run
```

The current validation implementation can also be reproduced with these logically equivalent records:

```python
# historical archive record
{"id": "...", "node_id": "...", ...}

# current live record
{"id": "...", "node_id": "...", ..., "agent_workspace_binding_id": None}
```

#### Root cause

[`_load_live_bundle_records()`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/services/retention/workflow_run/bundle_archive_maintenance.py#L806-L866) serializes the current ORM model through [`_row_to_dict()`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/services/retention/workflow_run/bundle_archive_maintenance.py#L1043-L1046), which includes all current columns. [`_validate_live_archive_subset()`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/services/retention/workflow_run/bundle_archive_maintenance.py#L868-L910) then computes a checksum over the complete live record dictionary.

The nullable [`workflow_node_executions.agent_workspace_binding_id`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/migrations/versions/2026_07_23_0203-f6e4c5686857_replace_agent_runtime_sessions_with_.py#L83-L87) column was added after the affected bundles had been archived.

#### Impact

- Prevents deletion of valid historical V2 archive bundles after nullable schema evolution.
- Can block an entire shard because maintenance stops at the first failed bundle.
- No data loss occurs: dry-run/fail-closed behavior correctly prevents deletion.

### ✔️ Expected Behavior

A live-only field absent from the archive schema should be tolerated only when its value is `NULL`. Any non-null live-only field must still block deletion.

A proposed fix in `_validate_live_archive_subset()` is to:

1. Detect fields present in the live record but absent from the corresponding archived record.
2. Remove them from the live comparison projection only if every such value is `NULL`.
3. Keep fail-closed behavior for:
- non-null live-only fields;
- archive/live ID differences;
- row-count differences;
- differences in any field represented in both records.

Regression coverage should include:

- Historical archive record lacks a newly added nullable column; live value is `NULL` → validation passes.
- Same setup, but live value is non-null → validation fails.
- Existing shared-field content mismatch → validation still fails.
- Live record contains an ID not present in the archive → validation still fails.

### ❌ Actual Behavior

The dry-run fails before deleting anything with:

```text
Live/archive subset content checksum mismatch for workflow_node_executions
```

The failure occurs even when:

- live and archive `workflow_node_executions` row counts match;
- their row ID sets match exactly;
- every shared field has the same value.

The only difference is that the current live ORM record contains:

```ini
agent_workspace_binding_id = NULL
```

while the historical Parquet record has no `agent_workspace_binding_id` key because it was archived before the column existed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.