HDFGroup / HDFGroup/hdf5

h5repack: SIGSEGV in Nbit filter processing when repacking datasets

Open
#6,492 2 comments 0 reactions 1 assignee Claimed by @mattjala View on GitHub
Dominant language
C
Stars
988
Forks
355
Avg merge
4d 2h
Merged PRs (30d)
12

Description

**Describe the bug**
Running `h5repack` on HDF5 files causes segmentation faults in the Nbit filter processing path during dataset copy. Two related PoC inputs trigger crashes in the same Nbit filter/chunk-read workflow: 1. One input reaches `H5Z__nbit_decompress_one_byte()` through `H5Z__nbit_decompress()` and `H5Z__filter_nbit()`, causing a SIGSEGV during Nbit decompression. 2. Another input crashes directly in `H5Z__filter_nbit()` with malformed filter parameters, including `cd_values=0x0`. Both crashes occur while `h5repack` is copying objects and reading dataset data through `H5Dread()` and `do_copy_objects()`.

**Steps to Reproduce:**

PoC inputs:

[OOB_Read.zip](https://github.com/user-attachments/files/29442523/OOB_Read.zip)
[OOB_Read_03.zip](https://github.com/user-attachments/files/29442522/OOB_Read_03.zip)

Command lines:

./h5repack -l CHUNK=2x2 OOB_Read out.h5
./h5repack -l COMPA OOB_Read_03 out.h5

**Expected behavior**
h5repack should exit cleanly with an error message, rather than crashing with SIGSEGV during filter processing or decompression.

**Platform (please complete the following information)**

- HDF5 version:

- Git branch: develop

- Commit: 8a2fb79

- OS and version:

- Ubuntu 22.04.5 LTS

- Compiler and version:

- clang: Ubuntu clang version 14.0.0-1ubuntu1.1

- gcc: gcc (Ubuntu 11.4.0-1ubuntu1~22.04.3) 11.4.0

- Build system and generator:

- CMake 4.3.2

- Generator: Unix Makefiles, if you used the default CMake generator

- Configure options:

- Not explicitly recorded in this run.

- If same as the previous report:

`-DHDF5_BUILD_TOOLS=ON -DHDF5_BUILD_EXAMPLES=OFF -DHDF5_BUILD_TESTING=OFF -DHDF5_BUILD_HL_LIB=ON -DBUILD_SHARED_LIBS=OFF -DBUILD_STATIC_LIBS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo`

- MPI library and version:

- Parallel HDF5 not enabled; no MPI library used.

**Additional context**
GDB output excerpt for the first PoC:

Program received signal SIGSEGV, Segmentation fault.

#0 0x00000000009e978e in H5Z__nbit_decompress_one_byte (
data=,
data_offset=194736,
k=1,
begin_i=3,
end_i=0,
buffer=0x289de490 ":\252\263ʫ>",
j=0x7ffeccdd4cf0,
buf_len=0x7ffeccdd4d10,
p=0x7ffeccdd4c90,
datatype_len=)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Znbit.c:1044
#1 H5Z__nbit_decompress_one_atomic (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Znbit.c:1100
#2 0x00000000009e6eea in H5Z__nbit_decompress (
data=0xbf3cc0 "",
buffer=0x0,
parms=,
d_nelmts=)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Znbit.c:1291
#3 H5Z__filter_nbit (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Znbit.c:956
#4 0x00000000009e34f8 in H5Z_pipeline (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Z.c:1451
#5 0x0000000000a43d19 in H5D__chunk_lock (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Dchunk.c:4855
#6 0x0000000000a34cd2 in H5D__chunk_read (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Dchunk.c:3177
#12 H5Dread (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5D.c:1049
#13 0x0000000000435229 in do_copy_objects (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack_copy.c:1053
#14 0x0000000000431bb7 in copy_objects (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack_copy.c:326
#15 0x00000000004553fa in h5repack (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack.c:53

GDB output excerpt for the second PoC:

Program received signal SIGSEGV, Segmentation fault.

#0 H5Z__filter_nbit (
flags=flags@entry=2305,
cd_nelmts=0,
cd_nelmts@entry=140728831307008,
cd_values=cd_values@entry=0x0,
nbytes=nbytes@entry=31,
buf_size=0x7ffdfbffee60,
buf=0x7ffdfbffeef0)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Znbit.c:935
#1 0x00000000009e34f8 in H5Z_pipeline (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Z.c:1451
#2 0x0000000000a43d19 in H5D__chunk_lock (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Dchunk.c:4855
#3 0x0000000000a34cd2 in H5D__chunk_read (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Dchunk.c:3177
#4 0x0000000000523f70 in H5D__read (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5Dio.c:398
#9 H5Dread (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/src/H5D.c:1049
#10 0x0000000000435229 in do_copy_objects (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack_copy.c:1053
#11 0x0000000000431bb7 in copy_objects (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack_copy.c:326
#12 0x00000000004553fa in h5repack (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack.c:53
#13 0x000000000045ab07 in main (...)
at /root/SpecDrive/benchmarks/hdf5/hdf5/tools/src/h5repack/h5repack_main.c:10

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.