alunduil / alunduil/zfs-replicate
generate() states the task graph it already computes
- 主要語言
- Python
- 星號
- 24
- 分支
- 6
- 平均合併
- 3 小時 11 分鐘
- 30 天內合併 PR
- 49
描述
## Summary
Have `generate()` return the ordering constraints alongside the tasks, so `schedule.dependencies()` and its six helpers stop reconstructing from names what the generator knew and discarded.
## Motivation
`zfs/replicate/task/schedule.py` is 254 lines, and roughly 106 of them are re-derivation rather than scheduling. `generate()` walks `local_snapshots` keyed by real `FileSystem` objects and emits tasks in an order that already satisfies ZFS, then encodes that order implicitly in list position and throws the structure away. `dependencies()` recovers it:
- `_data_set()` carries the comment "generate keys a SEND by the remote root rather than by its destination, so a send's data set comes from its snapshot instead of its task key." That is undoing a representation choice made one module over.
- `_nearest()` re-derives ancestry by splitting name strings on `/`, when `generate()` had the filesystem hierarchy in hand.
- `_group_by_data_set()`, `_index_by_data_set()`, and `_chain_edges()` re-index tasks that were emitted in groups.
The constraints themselves are cheap to state where they arise. `generate()` emits a `CREATE` and then that data set's `SEND`s adjacently, so the chain is known at emission; ancestry is known from the keys it is already iterating.
This does not change ADR-0001. The dispatcher, its main-thread loop over a bounded `ThreadPoolExecutor`, and the four ordering rules all stand. What moves is where the graph is built.
## Scope
- Return the tasks and their edges together from `generate()`, as a tuple or a small plan value.
- Build the edges where the structure is known, replacing the name-based re-derivation.
- Delete `dependencies()`, `_group_by_data_set()`, `_chain_edges()`, `_create_edges()`, `_destroy_edges()`, `_index_by_data_set()`, `_data_set()`, and `_nearest()`, leaving `schedule.py` as `dispatch()`.
- Keep `report()` working against whatever `generate()` returns, since `--dry-run` consumes it.
## Acceptance criteria
- [ ] `generate()` returns the tasks and the graph, and `schedule.py` no longer derives edges.
- [ ] The graph is identical to today's for the same inputs, shown by diffing the two over generated task lists rather than by the unit tests alone.
- [ ] `zfs_test/replicate_test/task_test/schedule_test.py`'s ordering tests keep asserting the four rules, wherever the rules end up living.
- [ ] No CLI change, and `--dry-run` output is unchanged.
## Out of scope
- Putting the edges on `Task` itself. It is a dataclass whose structural equality the scheduler tests compare, and a self-referential field breaks that and complicates `report()`.
- The dispatcher, the pool, and the `--jobs` flag.
- Replacing the graph with per-data-set streams. The CLI takes one `LOCAL_FS`, so a run is a single tree, and grouping by top-level filesystem yields one group and no parallelism for the case #394 exists to serve. Grouping by immediate child needs a barrier, which is ADR-0001's rejected option D. `filesystem.create` also walks the ancestry itself against a listing taken at entry, so concurrent sibling creates under a missing parent collide, which the parent-create edge serializes.
## Notes
- Target release: none. Internal, no CLI change.
- Blocked by #394, whose #513 introduces the module this reshapes.
- Surfaced reviewing #513, alongside #602.
貢獻指南
評估
這個 Issue 還沒有評估資料。