InsightSoftwareConsortium / InsightSoftwareConsortium/ITK

Register/disposition compiled-but-never-run tests (push line coverage past 90%)

Open
#6,370 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
1.7k
Forks
748
Avg merge
1d 1h
Merged PRs (30d)
64

Description

## Summary

While working toward pushing ITK's line coverage **past 90%** (the CDash `itk-coverage` build currently reports **89.51%**, ~760 covered lines short), I found a systemic source of silent coverage loss: **test source files that are compiled into their module test drivers but are never invoked by any `itk_add_test`**. They build, count as "present," but never run — so the code they would exercise shows as uncovered.

`itkOrientImageFilter` was the motivating example: 0% coverage on CDash despite **two** existing test files, purely because neither was registered with `itk_add_test`. A toolkit-wide scan found ~30 more.

This issue tracks registering (or otherwise dispositioning) each one. Verified history shows these are **move/reorg oversights**, not deliberate removals: none ever had an `itk_add_test` (`git log -G` = 0 historical invocations), none have GoogleTest replacements, and `Nonunit/IntegratedTest/test/CMakeLists.txt` even carried explicit `# missing itk_add_test() call` comments.

Detection method (reproducible)

For each `Modules/**/test/CMakeLists.txt`: strip `#`-comments, collect `itkFoo.cxx` source tokens that exist on disk, skip GoogleTest bodies (`TEST(` / `itkGTest.h`), require the file to define `int (int …)`, then flag if `` never appears in the CMake except as `.cxx`.

**Known blind spots (fix before re-running):** tests whose names are built in a `foreach`/variable (e.g. `itkDiscrete${loop_var}Curvature…`) are *not* dormant but look it; and a **commented-out** `itk_add_test` means *intentionally disabled*, not missing.

Done — registered, build + run locally, all pass (15 tests)

- **Filtering/ImageGrid:** `itkOrientImageFilterTest` (useImageDirection=0) + legacy `itkOrientImageFilterTest2`
- **Core/Common:** `itkObjectStoreTest`, `itkImageComputeOffsetAndIndexTest`, `itkImageRegionExplicitTest`, `itkMetaProgrammingLibraryTest`, `itkPromoteType`
- **Nonunit/IntegratedTest:** `itkFilterImageAddTest`, `itkReleaseDataFilterTest` (the two with `# missing itk_add_test()` comments)
- **IO/JPEG2000:** `itkJPEG2000ImageIOFactoryTest01`, `itkJPEG2000ImageIOTest00`
- **Segmentation/LevelSetsv4:** `itkLevelSetEquationChanAndVeseInternalTermTest`, `itkLevelSetEquationRegionTermTest`, `itkLevelSetEquationTermBaseTest`, `itkLevelSetEquationTermContainerTest`

Skipped — fail when run or need investigation (NOT just a registration)

- **Core/QuadEdgeMesh/`itkQuadEdgeMeshBasicLayerTest`** — runtime failure: `Erroneous GetDestination() on edge number 0. Was expecting 2 but got 0.` Behavior drift since it last ran.
- **Segmentation/LevelSetsv4/`itkLevelSetEquationChanAndVeseExternalTermTest`** — runs to "CV External term created" then returns `EXIT_FAILURE` (internal assertion/value issue).
- **Nonunit/Review/`itkWarpJacobianDeterminantFilterTest`** — `ITKReview` not built in the default configuration; needs the module enabled to verify.

Needs fixtures / parameters before registering

- **Filtering/LabelMap (3):** `itkStatistics{KeepNObjects,Opening,Relabel}LabelMapFilterTest1` — need 6 args (input + feature image + output + reverseOrdering + a valid statistics-attribute enum + count) and **no baseline exists** (would need one published).
- **IO/JPEG2000:** `itkJPEG2000ImageIOTest01/02/04` — need `.j2k` inputs + args.
- **IO/DCMTK/`itkDCMTKImageIOMultiFrameImageTest`** — needs a multi-frame DICOM fixture.
- **Video/BridgeVXL/`itkVXLVideoIOTest`** — needs VXL + a video fixture.
- **Core/ImageFunction / ImageGradient / Registration/Metricsv4 (4 `*SpeedTest`)** — benchmarks; likely intentionally not run in CI (confirm before enabling).

Scanner false positives (already running / intentionally disabled — leave alone)

- **Filtering/QuadEdgeMeshFiltering (4):** `itkDiscrete{Gaussian,Maximum,Mean,Minimum}CurvatureQuadEdgeMeshFilterTest` — already registered via `foreach(loop_var ${CURV_TESTS})`.
- **Filtering/IsotropicWavelets (2):** `itkZeroDCImageFilterTest`, `itkWaveletFrequencyFilterBankGeneratorDownsampleTest` — `itk_add_test` blocks are commented out (deliberate disable).
- **Bridge/VtkGlue/`runViewImage`** — commented-out interactive viewer (needs a display).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.