V2 archive delete fails when historical Parquet lacks a later-added nullable column
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
- [x] Please do not modify this template :) and fill in all the required fields.
### Dify version
Source-verified on `main` at `3765a41b8c9747ef881ac7ef59b36fb3755932d1`. The issue applies to deployments containing #39001 and the later nullable-column migration.
### Cloud or Self Hosted
Cloud
### Steps to reproduce
After [#39001](https://github.com/langgenius/dify/pull/39001), V2 workflow archive deletion validates that the remaining live rows are an unchanged subset of the archived Parquet rows before deleting anything.
This fails for archive bundles written before a nullable column was added to `workflow_node_executions`.
1. Use a historical V2 archive bundle whose `workflow_node_executions.parquet` was written before `agent_workspace_binding_id` existed.
2. Keep the corresponding live `workflow_node_executions` row after the migration, with `agent_workspace_binding_id = NULL`.
3. Verify that live and archive row counts and row ID sets match, and that all shared fields have identical values.
4. Run:
```shell
flask delete-archived-workflow-runs --target-month --dry-run
```
The current validation implementation can also be reproduced with these logically equivalent records:
```python
# historical archive record
{"id": "...", "node_id": "...", ...}
# current live record
{"id": "...", "node_id": "...", ..., "agent_workspace_binding_id": None}
```
#### Root cause
[`_load_live_bundle_records()`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/services/retention/workflow_run/bundle_archive_maintenance.py#L806-L866) serializes the current ORM model through [`_row_to_dict()`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/services/retention/workflow_run/bundle_archive_maintenance.py#L1043-L1046), which includes all current columns. [`_validate_live_archive_subset()`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/services/retention/workflow_run/bundle_archive_maintenance.py#L868-L910) then computes a checksum over the complete live record dictionary.
The nullable [`workflow_node_executions.agent_workspace_binding_id`](https://github.com/langgenius/dify/blob/3765a41b8c9747ef881ac7ef59b36fb3755932d1/api/migrations/versions/2026_07_23_0203-f6e4c5686857_replace_agent_runtime_sessions_with_.py#L83-L87) column was added after the affected bundles had been archived.
#### Impact
- Prevents deletion of valid historical V2 archive bundles after nullable schema evolution.
- Can block an entire shard because maintenance stops at the first failed bundle.
- No data loss occurs: dry-run/fail-closed behavior correctly prevents deletion.
### ✔️ Expected Behavior
A live-only field absent from the archive schema should be tolerated only when its value is `NULL`. Any non-null live-only field must still block deletion.
A proposed fix in `_validate_live_archive_subset()` is to:
1. Detect fields present in the live record but absent from the corresponding archived record.
2. Remove them from the live comparison projection only if every such value is `NULL`.
3. Keep fail-closed behavior for:
- non-null live-only fields;
- archive/live ID differences;
- row-count differences;
- differences in any field represented in both records.
Regression coverage should include:
- Historical archive record lacks a newly added nullable column; live value is `NULL` → validation passes.
- Same setup, but live value is non-null → validation fails.
- Existing shared-field content mismatch → validation still fails.
- Live record contains an ID not present in the archive → validation still fails.
### ❌ Actual Behavior
The dry-run fails before deleting anything with:
```text
Live/archive subset content checksum mismatch for workflow_node_executions
```
The failure occurs even when:
- live and archive `workflow_node_executions` row counts match;
- their row ID sets match exactly;
- every shared field has the same value.
The only difference is that the current live ORM record contains:
```ini
agent_workspace_binding_id = NULL
```
while the historical Parquet record has no `agent_workspace_binding_id` key because it was archived before the column existed.
Contributor guide
Assessment
This issue has not been assessed yet.