random test failures on older CPUs without SSE/AVX/AVX2/AVX512
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 5k
- Forks
- 622
- Avg merge
- 8h 37m
- Merged PRs (30d)
- 5
Description
Describe the bug
There are strange random test failures on older CPUs that don't have SSE/AVX/AVX2/AVX512.
To Reproduce
Steps to reproduce the behavior:
- Build SPTAG on a machine without AVX/AVX2/AVX512.
- Run
SPTAGTests - See error
Expected behavior
The tests should work.
Analysis
I think that because of the -mavx2 -mavx -msse -msse2 -mavx512f -mavx512bw -mavx512dq options in the DistanceUtils target_compile_options, the compiler is generating newer instructions in the DistanceUtils library and these are not run on older CPUs. Removing the options not supported by the CPU and deleting the functions using instructions that those options enable fixes this issue. So the cause is definitely the options being enabled.
Suggestions
On Linux you can use GCC function multi-versioning to get the compiler to automatically check the CPU at runtime and dispatch to the right functions.
Screenshots
Some examples of the random failures:
[1] Start invoking BuildTrees.
[1] BKTKmeansK: 3, BKTLeafSize: 6, Samples: 100, BKTLambdaFactor:-1.000000 TreeNumber: 1, ThreadNum: 2.
unknown location(0): fatal error: in "SSDServingTest/TestHeadUInt8L2DEFAULT": memory access violation at address: 0x00000000: no mapping at fault address
./Test/src/SSDServingTest.cpp(444): last checkpoint: "TestHeadUInt8L2DEFAULT" test entry
*** 1 failure is detected in the test module "Main"
[1] Parallel TpTree Partition done
[1] Build TPTree time (s): 4
[1] Processing Tree 0 0%
unknown location(0): fatal error: in "AlgoTest/KDTTest": signal: illegal operand; address of failing instruction: 0x559a5eccc130
./Test/src/AlgoTest.cpp(22): last checkpoint
*** 1 failure is detected in the test module "Main"
Desktop:
- OS: Debian
- Browser None
- Version 12 (bookworm)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the DistanceUtils target_compile_options and run SPTAGTests on a CPU without the listed SSE/AVX extensions, focusing on the failures in SSDServingTest and AlgoTest. The work is done when the test suite no longer executes unsupported instructions or crashes on older CPUs while retaining supported optimized paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp
- Domain
- build-system, performance, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100