HDFGroup / HDFGroup/hdf5

Investigate and Enable Compiler Security Hardening Flags for HDF5 Binaries

Open
#6,203 0 comments 1 reaction 1 assignee Claimed by @brtnfld View on GitHub
Component - Build HDFG-internal
Dominant language
C
Stars
988
Forks
355
Avg merge
4d 2h
Merged PRs (30d)
12

Description

**Summary**
As part of modern software security best practices, it is recommended to compile C/C++ libraries with hardening flags. These flags enable compiler and linker features that mitigate common vulnerabilities such as stack buffer overflows, format string attacks, and Return-Oriented Programming (ROP).

This issue tracks the investigation into whether current HDF5 binary releases should be built with these protections, and proposes integrating them into CMake by default for release builds.

**Goals**
1. **Verify Status:** Determine which security flags are currently neded.
2. **Enhance Security:** Update the build configuration to include standard hardening flags
3. **Validation:** Ensure these flags do not introduce regressions or unacceptable performance penalties.

**Proposed Flags for Investigation**
We should investigate enabling the following flags (or their MSVC equivalents on Windows) for release builds:

* **Stack Smashing Protection:**
* `-fstack-protector-strong`: Emits extra code to check for buffer overflows, protecting functions with local arrays or references to local frame addresses.
* **Buffer Overflow Detection:**
* `-D_FORTIFY_SOURCE=2` (or `=3`): Performs compile-time and run-time checks for buffer overflows in common string and memory functions.
* **Position Independent Execution (ASLR support):**
* `-fPIE` (Compile) / `-pie` (Link): Generates position-independent code for executables, allowing the kernel to randomize the memory layout (ASLR) more effectively.
* **Relocation Read-Only (RELRO):**
* `-Wl,-z,relro`: Marks the Global Offset Table (GOT) as read-only after the linker resolves symbols, preventing GOT overwrite attacks.
* `-Wl,-z,now`: Resolves all symbols at startup, allowing the entire PLT/GOT to be marked read-only (Full RELRO).
* **Stack Clash Protection:**
* `-fstack-clash-protection`: Prevents the stack from colliding with the heap.

**Action Items**
* [ ] Run `checksec --file ` on current release binaries to audit current hardening status.
* [ ] Review the CMake implementation and Github Actions to see if these flags can be optionally enabled or set as default for `Release` builds.
* [ ] Measure the performance impact (if any) on standard HDF5 benchmarks to ensure acceptable overhead.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.