InsightSoftwareConsortium / InsightSoftwareConsortium/ITK
Register/disposition compiled-but-never-run tests (push line coverage past 90%)
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 748
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 64
Description
## Summary
While working toward pushing ITK's line coverage **past 90%** (the CDash `itk-coverage` build currently reports **89.51%**, ~760 covered lines short), I found a systemic source of silent coverage loss: **test source files that are compiled into their module test drivers but are never invoked by any `itk_add_test`**. They build, count as "present," but never run — so the code they would exercise shows as uncovered.
`itkOrientImageFilter` was the motivating example: 0% coverage on CDash despite **two** existing test files, purely because neither was registered with `itk_add_test`. A toolkit-wide scan found ~30 more.
This issue tracks registering (or otherwise dispositioning) each one. Verified history shows these are **move/reorg oversights**, not deliberate removals: none ever had an `itk_add_test` (`git log -G` = 0 historical invocations), none have GoogleTest replacements, and `Nonunit/IntegratedTest/test/CMakeLists.txt` even carried explicit `# missing itk_add_test() call` comments.
Detection method (reproducible)
For each `Modules/**/test/CMakeLists.txt`: strip `#`-comments, collect `itkFoo.cxx` source tokens that exist on disk, skip GoogleTest bodies (`TEST(` / `itkGTest.h`), require the file to define `int (int …)`, then flag if `` never appears in the CMake except as `.cxx`.
**Known blind spots (fix before re-running):** tests whose names are built in a `foreach`/variable (e.g. `itkDiscrete${loop_var}Curvature…`) are *not* dormant but look it; and a **commented-out** `itk_add_test` means *intentionally disabled*, not missing.
Done — registered, build + run locally, all pass (15 tests)
- **Filtering/ImageGrid:** `itkOrientImageFilterTest` (useImageDirection=0) + legacy `itkOrientImageFilterTest2`
- **Core/Common:** `itkObjectStoreTest`, `itkImageComputeOffsetAndIndexTest`, `itkImageRegionExplicitTest`, `itkMetaProgrammingLibraryTest`, `itkPromoteType`
- **Nonunit/IntegratedTest:** `itkFilterImageAddTest`, `itkReleaseDataFilterTest` (the two with `# missing itk_add_test()` comments)
- **IO/JPEG2000:** `itkJPEG2000ImageIOFactoryTest01`, `itkJPEG2000ImageIOTest00`
- **Segmentation/LevelSetsv4:** `itkLevelSetEquationChanAndVeseInternalTermTest`, `itkLevelSetEquationRegionTermTest`, `itkLevelSetEquationTermBaseTest`, `itkLevelSetEquationTermContainerTest`
Skipped — fail when run or need investigation (NOT just a registration)
- **Core/QuadEdgeMesh/`itkQuadEdgeMeshBasicLayerTest`** — runtime failure: `Erroneous GetDestination() on edge number 0. Was expecting 2 but got 0.` Behavior drift since it last ran.
- **Segmentation/LevelSetsv4/`itkLevelSetEquationChanAndVeseExternalTermTest`** — runs to "CV External term created" then returns `EXIT_FAILURE` (internal assertion/value issue).
- **Nonunit/Review/`itkWarpJacobianDeterminantFilterTest`** — `ITKReview` not built in the default configuration; needs the module enabled to verify.
Needs fixtures / parameters before registering
- **Filtering/LabelMap (3):** `itkStatistics{KeepNObjects,Opening,Relabel}LabelMapFilterTest1` — need 6 args (input + feature image + output + reverseOrdering + a valid statistics-attribute enum + count) and **no baseline exists** (would need one published).
- **IO/JPEG2000:** `itkJPEG2000ImageIOTest01/02/04` — need `.j2k` inputs + args.
- **IO/DCMTK/`itkDCMTKImageIOMultiFrameImageTest`** — needs a multi-frame DICOM fixture.
- **Video/BridgeVXL/`itkVXLVideoIOTest`** — needs VXL + a video fixture.
- **Core/ImageFunction / ImageGradient / Registration/Metricsv4 (4 `*SpeedTest`)** — benchmarks; likely intentionally not run in CI (confirm before enabling).
Scanner false positives (already running / intentionally disabled — leave alone)
- **Filtering/QuadEdgeMeshFiltering (4):** `itkDiscrete{Gaussian,Maximum,Mean,Minimum}CurvatureQuadEdgeMeshFilterTest` — already registered via `foreach(loop_var ${CURV_TESTS})`.
- **Filtering/IsotropicWavelets (2):** `itkZeroDCImageFilterTest`, `itkWaveletFrequencyFilterBankGeneratorDownsampleTest` — `itk_add_test` blocks are commented out (deliberate disable).
- **Bridge/VtkGlue/`runViewImage`** — commented-out interactive viewer (needs a display).
Contributor guide
Assessment
This issue has not been assessed yet.