CI has no Windows coverage: hadoop dependency bumps can silently break Windows development
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
Add a small Windows CI leg that exercises the OS-sensitive paths, so dependency bumps that break Windows development fail in the PR instead of on a dev machine weeks later.
**Motivation:** #6487 was a Windows-only regression that CI could not catch: the hadoop-common bumps #6201 (3.3.1 → 3.4.3) and #6227 (→ 3.5.0) made every local Iceberg write require `winutils.exe`, breaking all workflow executions on Windows dev machines — while `ubuntu-latest` and `macos-latest` CI stayed green. See the discussion in #6488.
A minimal starting point:
| | |
|---|---|
| Runner | `windows-latest` |
| Scope | `WorkflowCore` unit tests (or just the file-system-sensitive specs, e.g. `IcebergUtilSpec` + `WinutilsFreeLocalFileSystemSpec`) |
| Would it have caught #6487? | Yes — the regression test added in #6488 fails on a stock Windows runner without the fix |
Notes / open questions:
- A full `build / amber` Windows leg is likely too heavy (Windows runners are slower and billed at 2x; amber integration tests spawn Python workers via `signal.SIGKILL`, which does not exist on Windows), so scoping to unit tests of the storage/file-system layer keeps the cost low.
- A Windows leg would also cover the Windows-only branches that Linux-based Codecov runs cannot execute.
- Could run on `main` + dependency-labeled PRs only, if PR latency is a concern.
### Task Type
- [ ] Refactor / Cleanup
- [x] DevOps / Deployment / CI
- [x] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
Contributor guide
Research direction
Start by locating the existing CI workflow and the WorkflowCore unit-test configuration, then run the file-system-sensitive IcebergUtilSpec and WinutilsFreeLocalFileSystemSpec on a Windows runner. Done means a lightweight windows-latest CI leg runs the selected tests reliably and catches the regression described in #6488 without attempting the incompatible full amber suite.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, scala
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100