Returns ERROR_INVALID_BLOCK on large functions with __chkstk on it.
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 6.4k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
Latest version as for today(Version 4.0.1).
Trying to hook to a large function, looking at the disassembly:
9: void function()
10: {
00007FF6E296E010 B8 C8 6C 00 00 mov eax,6CC8h
00007FF6E296E015 E8 A3 3B FE FF call __chkstk (07FF6E2951BBDh)
00007FF6E296E01A 48 2B E0 sub rsp,rax
I have the same project built at home and the __chkstk is not there, deours hook without issues. At another PC with the same compiler, it builds adding that instruction and detour fails to hook.
I assume the block is too small thanks to that call to __chkstk. I think it should be supported, but still, please let me know if this is an issue or if it's expected.
Building with VS2022 target x64, compiler options: /MD /Zi /O2 /Ob1 /DNDEBUG
Thanks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the ERROR_INVALID_BLOCK failure with Detours 4.0.1 using VS2022 x64 and the reported /MD /Zi /O2 /Ob1 /DNDEBUG options, comparing builds with and without __chkstk. Use the disassembly in the report as the initial scope; done means establishing whether large functions containing __chkstk can be hooked or documenting why the behavior is expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100