mandiant / mandiant/capa-rules

[64-bit execution via heavens gate] current rule misses obfuscated variants

Open
#1,096 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

false negative
Dominant language
No language data
Stars
736
Forks
245
Avg merge
4d 53m
Merged PRs (30d)
2

Description

Summary

While analyzing a sample, I observed a false negative when the code segment (CS) selector value is computed at runtime
using arithmetic operations. The current static rule relies on detecting the constant 0x33 directly (for example via a push or mov), which means obfuscated variants that compute the same value dynamically are not detected.

  - instruction:
            - mnemonic: push
            - number: 0x33
          - instruction:
            - mnemonic: mov
            - number: 0x33

Examples

The sample I am analysing does the following:

push 3
pop eax
shl eax, 4
add al, 3
push eax
push 0
retf

In this binary, the Heaven’s Gate transition itself is implemented as runtime-executed shellcode rather than a statically visible code sequence.

Both approaches eventually place 0x33 into the CS register and perform a Heaven’s Gate transition. The second version is simply obfuscated, which causes the current capa rule to miss it. Enumerating all possible arithmetic ways to compute the same value does not seem practical.

The sample was one I developed for a CTF challenge, please have a look:
Sample: https://drive.proton.me/urls/ZD2V8HKZCM#fSqFaLtxBubo
Password: infected

Link to the CTF challenge writeup: https://github.com/Cryptonite-MIT/niteCTF-2025/tree/main/rev/lament_of_the_seraph/solution

Possible improvements

I understand that determining the value of the CS register at runtime is out of scope for static capa analysis. However, the presence of Heaven’s Gate may still be inferred indirectly. One possible way to make the rule more robust could be to detect the use of 64-bit–only instructions inside a 32-bit binary, which strongly suggests a transition into 64-bit mode.

    - arch: i386
        - or:
          - mnemonic: syscall
          - mnemonic: sysret
          - mnemonic: swapgs
          - mnemonic: retfq
          - mnemonic: lretq
          - mnemonic: cdqe
          - mnemonic: movsxd

Additional context

This appears to be a limitation of static pattern-based detection when values are constructed dynamically at runtime. I wanted to raise this for discussion and to understand whether this is an accepted limitation or if similar heuristics have been considered before.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the current YAML rule pattern shown in the issue, then reproduce the miss with the provided assembly sequence and linked CTF writeup. A concrete resolution is not specified, so determine whether a feasible heuristic for obfuscated Heaven’s Gate transitions is acceptable and define tests for the chosen behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
yaml
Domain
reverse-engineering, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.