Undefined behavior in PACKEDANCESTRYMAP hash code
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 207
- Forks
- 61
- PR merge metrics
- No merged PRs in 30d
Description
The hasharr and hashit functions in src/admutils.c, which are used to compute the .snp / .ind hash values stored in the PACKEDANCESTRYMAP header, use signed integers that usually overflow in the middle of the computation. While unsigned-integer overflow is defined to "wrap around" in C, signed-integer overflow is "undefined behavior": compilers are allowed to assume signed-integer overflow never happens, and generate code that is correct in the no-overflow case but does weird things on overflow.
I'm guessing no existing compiler has actually done anything weird here, but it would be good to eliminate the possibility.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/admutils.c at the hasharr and hashit functions, then trace how they produce the .snp and .ind hash values stored in the PACKEDANCESTRYMAP header. Check the integer operations involved and verify that the resulting hash values remain compatible after the undefined behavior is removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100