Tracking: audit of broken/skipped/disabled tests in the test suite
- Dominant language
- Julia
- Stars
- 258
- Forks
- 138
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 3
Description
This is a tracking issue cataloguing the broken/skipped/disabled tests in the test suite that **do not** have a corresponding open issue, with their stated reason. It is meant as an umbrella to be broken down into individual issues/PRs later.
Audited against the current `master` and the open issue list as of 2026-06-09.
> **#706 (merged 2026-06-09)** fixed a real, backend-independent `hardσ` gradient correctness bug and resolved several catalogued items: the `conv.jl` gradtest "hang", the `maxpool` `spatial_rank==2` skip, the Metal `hardσ`/`leakyrelu` breakage, and the stale `∇depthwiseconv_data` FIXME. Those rows have been removed below; only the still-open, untracked items remain.
## Individually broken/skipped tests
| Test site | What's broken/skipped | Reason given | Notes |
|---|---|---|---|
| `test/dropout.jl` | `@test_broken Zygote.hessian(f1, …)` | forward-over-reverse 2nd-order AD of `dropout` | AD limitation |
| `test/conv.jl` | `broken=flag_` on `∇conv_filter_direct!` (stride, rank ∈ (1,3), alpha/beta) and `∇conv_data_direct!` (dilation, rank 3, alpha/beta) | — | #706 identified this as a genuine `conv_direct!` accumulation bug (the `beta≠0` path mis-accumulates). **Needs its own issue/fix — highest-value remaining item.** |
| `test/bias_act.jl` | `@test_skip` 2nd-order AD through `bias_act!(relu,…)` | global variable `b` causes a Zygote `foreigncall` error | Zygote `accum_global` limitation |
| `test/ext_cuda/conv.jl` | `checkgrad=false # TODO` on conv `alpha=2.0` scaling | gradient checks disabled for the scaling path | likely the same `conv_direct!`/scaling-path bug as above |
## FIXME / TODO comments where the test still runs (not actually skipped)
| Test site | Note |
|---|---|
| `test/ext_amdgpu/batched_mul.jl` | `# FIXME same but with (1, 3, 2) errors` — the `(2,1,3)` permutation is tested; the `(1,3,2)` case is just described, not run |
| ~~`test/testsuite/spectral.jl`~~ | `@inferred stft(...)` commented out — "TODO fix type stability for `pad_reflect`" |
| `test/testsuite/upsample.jl` | `T = Float32 # TODO test against all supported eltypes` |
## Suggested follow-up
- Open a dedicated issue for the `conv_direct!` alpha/beta accumulation bug (genuine correctness bug, not just a test gap).
- Break the remaining untracked items into individual issues (or fix-and-remove for the stale ones).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.