alunduil / alunduil/zfs-replicate
Operators can replicate independent datasets in parallel
- Langage dominant
- Python
- Étoiles
- 24
- Forks
- 6
- Merge moyen
- 3 h 11 min
- PR mergées (30 j)
- 49
Description
## User story
As an **operator with many independent datasets**, I want to **replicate them in parallel** so that **a replication run completes in roughly the time of the slowest dataset rather than the sum of all datasets**.
## Why
`task/execute.py:20-42` iterates filesystems sequentially. Top-level filesystems are independent — their `send`/`receive` pipes do not share ordering constraints. #3 has been open since 2018. With a simple `ThreadPoolExecutor` and a user-tunable concurrency limit, this is a low-risk win.
## Acceptance criteria
- [ ] A new CLI flag `-j N / --jobs N` exists, with default `1` (preserves current behavior).
- [ ] When `N > 1`, tasks grouped by top-level filesystem run on a bounded `ThreadPoolExecutor`.
- [ ] Child datasets still run after their parent CREATE completes (existing sort by name depth continues to be honored).
- [ ] Operational output (see the logging issue) does not interleave mid-line; each task reports as a single log record.
- [ ] Failures in one dataset do not silently stop others; the run continues and exits non-zero at the end if any task failed.
- [ ] `--help` describes the flag.
- [ ] Tests cover the sequential path (`--jobs 1`) and the parallel path (`--jobs 4`) producing the same final state.
- [ ] CHANGELOG entry under "Added".
## Out of scope
- Parallel streams *within* a single dataset.
- Multi-host parallelism.
## Notes
- Target release: **v4.3.0** (minor — additive flag)
- Depends on the "operational output flows through the logging module" chore to keep logs readable under parallelism.
- Closes the user need from #3.
- Source: modernization assessment §3.5.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.