Vector35 / Vector35/binaryninja-api

Visually distinguish call sites with custom calling conventions

Open
#3,163 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Component: UI Effort: Low Impact: Low
Dominant language
C++
Stars
1.3k
Forks
298
Avg merge
5d 5h
Merged PRs (30d)
19

Description

Version and Platform (required):

  • Binary Ninja Version: 3474-dev
  • OS: Windows
  • OS Version: 11
  • CPU Architecture: x64

Bug Description:
Call sites of functions with custom calling conventions are not distinguishable from bugs. For example, opening notepad.exe gives me the following:

       140001b60  _start:
   0 @ 140001b64  __security_init_cookie()
   1 @ 140001b6d  uint64_t rbx
   2 @ 140001b6d  return sub_1400019d0(rbx) __tailcall

Does sub_1400019d0 take the register rbx via a custom calling convention, or is the entry value of rbx spilled to the stack and passed by value here? While _start makes it somewhat obvious what is going on, this can happen elsewhere.

Steps To Reproduce:
Please provide all steps required to reproduce the behavior:

  1. Open any windows executable
  2. Navigate to _start
  3. See error

Expected Behavior:
I expect pseudo-C's semantics to match what is displayed. Specifically, imagine the use case of binja as a manual audit tool for uninitialized stack usage.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report by opening a Windows executable, navigating to _start, and inspecting the pseudo-C around sub_1400019d0. Trace how custom calling conventions and register arguments are displayed; done means call sites clearly distinguish a custom convention from a spilled entry register while preserving pseudo-C semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.