Vector35 / Vector35/binaryninja-api

Analysis is non-deterministic

Open
#3,852 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Component: Core Effort: Medium Impact: Medium
Dominant language
C++
Stars
1.3k
Forks
298
Avg merge
5d 5h
Merged PRs (30d)
19

Description

Version and Platform (required):

  • Binary Ninja Version: [e.g. 3.3.3996]
  • OS: Windows 11
  • OS Version: 22H2
  • CPU Architecture x64

Bug Description:
Different analyses introduce non-determinism into Binary Ninja. There appear to be two primary sources of the differences:

  1. Comparisons show up as ">" or "f>" inconsistently in MLIL.
  2. Phi variable numbers are inconsistent in MLIL_SSA

These differences carry through to HLIL analysis.

Steps To Reproduce:
Run my test_binja_consistency.py script that runs analysis multiple times until it sees a difference. This script displays differences at several different IL levels, and also outputs {il_form}.baseline and {il_form}.other files for use with a better diff utility.

Expected Behavior:
Consistent results when running analysis repeatedly.

Additional Information:

I am attaching 3 files:

  1. test_binja_consistency.py - The test script I have been using to narrow down non-determinism.
  2. il_diffs.zip - output from some of my runs of the script. The .baseline and .other files can be diffed using any diff utility (e.g. diff mlil.*, colordiff mlil.*, or vim -d mlil.*)
  3. df - the standard linux "df" binary that I have been testing against.

test_binja_consistency.zip
il_diffs.zip
df.zip

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the attached test_binja_consistency.py script repeatedly and compare the generated MLIL and MLIL_SSA baseline/other files, using il_diffs.zip and df.zip as reference inputs. Done means repeated analysis produces consistent comparison forms and Phi variable numbers through MLIL, MLIL_SSA, and HLIL.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
reverse-engineering
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.