bobluppes / bobluppes/graaf

Add sanitizer (ASan/UBSan/TSan) and fuzz testing to CI

Open
#326 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
413
Forks
67
Avg merge
7h 24m
Merged PRs (30d)
53

Description

## Summary

CI does not run the test suite under any sanitizer (ASan, UBSan, TSan) or with any fuzz testing. The only dynamic-analysis signal currently produced is line/branch coverage.

## Current state

- Neither [.github/workflows/main-ci.yml](../blob/main/.github/workflows/main-ci.yml) nor [.github/workflows/pull-request.yml](../blob/main/.github/workflows/pull-request.yml) pass any `-fsanitize=...` flags, and `CMakeLists.txt` has no `ENABLE_SANITIZER`-style option (compare to the existing `ENABLE_COVERAGE` option used for the coverage job).
- There is no fuzzing harness anywhere in the repo (e.g. nothing under `test/` or a dedicated `fuzz/` directory targets libFuzzer/AFL-style entry points), despite the library including a hand-rolled DOT-format parser/serializer ([include/graaflib/io/dot.h](../blob/main/include/graaflib/io/dot.h)) that is a natural candidate for fuzzing since it processes external, potentially untrusted input.
- Both CI workflows only build in `Debug` with plain compiler flags; there's no `RelWithDebInfo`+sanitizer configuration exercised anywhere.

## Why this matters

95%+ line coverage (the project's own [codecov.yml](../blob/main/codecov.yml) target) tells you code was *executed*, not that it's free of undefined behavior, memory errors, or data races — those require dedicated tooling. For a header-only library that will be compiled directly into many different downstream binaries (rather than shipped as a pre-built, independently-hardened artifact), catching UB/memory issues before release is more important than for a typical application, since every consumer inherits any latent bug at their own optimization levels and platforms. Given the library ships a hand-written parser that accepts external input, this is a concrete, not just theoretical, risk area.

## Suggested resolution

- Add a CMake option (e.g. `ENABLE_SANITIZER=address|undefined|thread`) that adds the corresponding `-fsanitize=...` flags to the test build.
- Add CI jobs that build and run the existing test suite under ASan+UBSan, and separately under TSan.
- Add a small libFuzzer (or similar) harness for the DOT parser and run it for a bounded time in CI (or as a periodic/scheduled job), seeded with the existing test fixtures.

## Acceptance criteria

- [ ] CI runs the test suite under ASan+UBSan on every PR (or at minimum on merges to `main`).
- [ ] CI runs the test suite under TSan (relevant once/if thread-safety guarantees are documented, see the related issue).
- [ ] A fuzz target exists for the DOT parser and runs regularly, with any findings triaged.

Contributor guide

Open the contributing guide

Research direction

The issue points to .github/workflows/main-ci.yml and .github/workflows/pull-request.yml for CI, and CMakeLists.txt for build configuration. The DOT parser is in include/graaflib/io/dot.h. Start by adding an ENABLE_SANITIZER CMake option, then integrate sanitizer flags into the CI workflows. For fuzzing, create a libFuzzer harness in a new directory like fuzz/. Done looks like CI jobs passing with sanitizers enabled and a fuzz target running.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, github-actions
Domain
ci-cd, security, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.