FluxML / FluxML/NNlib.jl

Tracking: audit of broken/skipped/disabled tests in the test suite

Open
#702 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
258
Forks
138
Avg merge
2d 1h
Merged PRs (30d)
3

Description

This is a tracking issue cataloguing the broken/skipped/disabled tests in the test suite that **do not** have a corresponding open issue, with their stated reason. It is meant as an umbrella to be broken down into individual issues/PRs later.

Audited against the current `master` and the open issue list as of 2026-06-09.

> **#706 (merged 2026-06-09)** fixed a real, backend-independent `hardσ` gradient correctness bug and resolved several catalogued items: the `conv.jl` gradtest "hang", the `maxpool` `spatial_rank==2` skip, the Metal `hardσ`/`leakyrelu` breakage, and the stale `∇depthwiseconv_data` FIXME. Those rows have been removed below; only the still-open, untracked items remain.

## Individually broken/skipped tests

| Test site | What's broken/skipped | Reason given | Notes |
|---|---|---|---|
| `test/dropout.jl` | `@test_broken Zygote.hessian(f1, …)` | forward-over-reverse 2nd-order AD of `dropout` | AD limitation |
| `test/conv.jl` | `broken=flag_` on `∇conv_filter_direct!` (stride, rank ∈ (1,3), alpha/beta) and `∇conv_data_direct!` (dilation, rank 3, alpha/beta) | — | #706 identified this as a genuine `conv_direct!` accumulation bug (the `beta≠0` path mis-accumulates). **Needs its own issue/fix — highest-value remaining item.** |
| `test/bias_act.jl` | `@test_skip` 2nd-order AD through `bias_act!(relu,…)` | global variable `b` causes a Zygote `foreigncall` error | Zygote `accum_global` limitation |
| `test/ext_cuda/conv.jl` | `checkgrad=false # TODO` on conv `alpha=2.0` scaling | gradient checks disabled for the scaling path | likely the same `conv_direct!`/scaling-path bug as above |

## FIXME / TODO comments where the test still runs (not actually skipped)

| Test site | Note |
|---|---|
| `test/ext_amdgpu/batched_mul.jl` | `# FIXME same but with (1, 3, 2) errors` — the `(2,1,3)` permutation is tested; the `(1,3,2)` case is just described, not run |
| ~~`test/testsuite/spectral.jl`~~ | `@inferred stft(...)` commented out — "TODO fix type stability for `pad_reflect`" |
| `test/testsuite/upsample.jl` | `T = Float32 # TODO test against all supported eltypes` |

## Suggested follow-up

- Open a dedicated issue for the `conv_direct!` alpha/beta accumulation bug (genuine correctness bug, not just a test gap).
- Break the remaining untracked items into individual issues (or fix-and-remove for the stale ones).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.