avast / avast/retdec

Decompile of Kernel Binary Failed

Open
#566 10 comments 0 reactions 0 assignees View on GitHub
C-bin2llvmir P-run T-format-macho
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Tried to decompile a kernel binary but failed.
```

$ /Users/user/Downloads/retdec/bin/retdec-decompiler.py /Users/user/Downloads/retdec_output/kernel
##### Checking if file is a Mach-O Universal static library...

##### Checking if file is an archive...
RUN: /Users/user/Downloads/retdec/bin/retdec-ar-extractor /Users/user/Downloads/retdec_output/kernel --arch-magic
Not an archive, going to the next step.

##### Gathering file information...
RUN: /Users/user/Downloads/retdec/bin/retdec-fileinfo -c /Users/user/Downloads/retdec_output/kernel.json --similarity /Users/user/Downloads/retdec_output/kernel --no-hashes=all --crypto /Users/user/Downloads/retdec/bin/../share/retdec/support/generic/yara_patterns/signsrch/signsrch.yara --crypto /Users/user/Downloads/retdec/bin/../share/retdec/support/generic/yara_patterns/signsrch/signsrch.yarac --max-memory-half-ram
Input file : /Users/user/Downloads/retdec_output/kernel
File format : Mach-O
File class : 64-bit
File type : Executable file
Architecture : x86-64
Endianness : Little endian
Entry point address : 0xffffff8000192000
Entry point offset : 0xb24000
Entry point section name : __text
Entry point section index: 20
Bytes on entry point : 89c7bc00101900b8403019000f0110b80000100001000180f80f0000ba001010000102014208014210014218b8200000000f
Detected tool : UPX (3.91 [LZMA]) (packer), 12 from 58 significant nibbles (20.6897%)

##### Trying to unpack /Users/user/Downloads/retdec_output/kernel into /Users/user/Downloads/retdec_output/kernel-unpacked.tmp by using generic unpacker...
RUN: /Users/user/Downloads/retdec/bin/retdec-unpacker /Users/user/Downloads/retdec_output/kernel -o /Users/user/Downloads/retdec_output/kernel-unpacked.tmp --max-memory-half-ram
[ERROR] [UPX] Unsupported unpacking stub detected.
##### Unpacking by using generic unpacker: nothing to do
##### 'upx' not available: nothing to do

##### Decompiling /Users/user/Downloads/retdec_output/kernel into /Users/user/Downloads/retdec_output/kernel.bc...
RUN: /Users/user/Downloads/retdec/bin/retdec-bin2llvmir -provider-init -decoder -verify -x87-fpu -main-detection -idioms-libgcc -inst-opt -cond-branch-opt -syscalls -stack -constants -param-return -local-vars -inst-opt -simple-types -generate-dsm -remove-asm-instrs -class-hierarchy -select-fncs -unreachable-funcs -inst-opt -x86-addr-spaces -value-protect -instcombine -tbaa -targetlibinfo -basicaa -domtree -simplifycfg -domtree -early-cse -lower-expect -targetlibinfo -tbaa -basicaa -globalopt -mem2reg -instcombine -simplifycfg -basiccg -domtree -early-cse -lazy-value-info -jump-threading -correlated-propagation -simplifycfg -instcombine -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -instcombine -scalar-evolution -loop-simplifycfg -loop-simplify -aa -loop-accesses -loop-load-elim -lcssa -indvars -loop-idiom -loop-deletion -memdep -gvn -memdep -sccp -instcombine -lazy-value-info -jump-threading -correlated-propagation -domtree -memdep -dse -dce -bdce -adce -die -simplifycfg -instcombine -strip-dead-prototypes -globaldce -constmerge -constprop -instnamer -domtree -instcombine -instcombine -tbaa -targetlibinfo -basicaa -domtree -simplifycfg -domtree -early-cse -lower-expect -targetlibinfo -tbaa -basicaa -globalopt -mem2reg -instcombine -simplifycfg -basiccg -domtree -early-cse -lazy-value-info -jump-threading -correlated-propagation -simplifycfg -instcombine -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -instcombine -scalar-evolution -loop-simplifycfg -loop-simplify -aa -loop-accesses -loop-load-elim -lcssa -indvars -loop-idiom -loop-deletion -memdep -gvn -memdep -sccp -instcombine -lazy-value-info -jump-threading -correlated-propagation -domtree -memdep -dse -dce -bdce -adce -die -simplifycfg -instcombine -strip-dead-prototypes -globaldce -constmerge -constprop -instnamer -domtree -instcombine -inst-opt -simple-types -stack-ptr-op-remove -idioms -global-to-local -dead-global-assign -instcombine -inst-opt -idioms -phi2seq -value-protect -disable-inlining -disable-simplify-libcalls -config-path /Users/user/Downloads/retdec_output/kernel.json -max-memory-half-ram -o /Users/user/Downloads/retdec_output/kernel.bc
Running phase: Initialization ( 0.02s )
Running phase: LLVM ( 0.03s )
Running phase: Providers initialization ( 0.04s )
Running phase: Input binary to LLVM IR decoding ( 1.13s )
Running phase: LLVM ( 110.33s )
Running phase: x87 fpu register analysis ( 112.39s )
Running phase: Main function identification optimization ( 112.94s )
Running phase: Libgcc idioms optimization ( 112.98s )
Running phase: LLVM instruction optimization ( 112.98s )
Running phase: Conditional branch optimization ( 114.02s )
Running phase: Syscalls optimization ( 124.53s )
Running phase: Stack optimization ( 124.53s )
Running phase: Constants optimization ( 151.87s )
Running phase: Function parameters and returns optimization ( 486.60s )
Running phase: Register localization optimization ( 522.07s )
Running phase: LLVM instruction optimization ( 537.23s )
Running phase: Simple types recovery optimization ( 538.59s )
Running phase: Disassembly generation ( 567.44s )
Running phase: Assembly mapping instruction removal ( 582.34s )
Running phase: C++ class hierarchy optimization ( 593.89s )
Running phase: Selected functions optimization ( 595.78s )
Running phase: Unreachable functions optimization ( 595.78s )
Running phase: LLVM instruction optimization ( 596.71s )
Running phase: x86 address spaces optimization ( 598.19s )
Running phase: Value protection optimization ( 598.85s )
Running phase: LLVM ( 601.92s )
Running phase: LLVM instruction optimization ( 2956.64s )
Running phase: Simple types recovery optimization ( 2958.04s )
Running phase: Stack pointer operations optimization ( 2958.09s )
Running phase: Instruction idioms optimization ( 2959.92s )
Running phase: Global to local optimization ( 2963.71s )
Error: Decompilation to LLVM IR failed
```

Can the program decompile kernel binaries?

If I ``file`` the binary it returns that it is a ``Mach-O 64-bit executable x86_64``.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the failure with retdec-decompiler.py using the Mach-O kernel binary, then inspect the retdec-unpacker and retdec-bin2llvmir stages shown in the log. Determine whether the unsupported UPX stub or LLVM IR conversion causes the failure; done means documenting supported kernel-binary behavior or producing a successful decompilation for this input.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems, reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.