[WARP] relocatable_regions is very expensive with a large number of segments / sections

Open
#8,081 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
cpp, rust
Domain
performance, reverse-engineering

Research direction

Start in plugins/warp/src/lib.rs at relocatable_regions, especially the linked lines calling BinaryView::GetSectionsAt and BinaryViewExt::segments. Reproduce with a large iOS shared cache using the listed load and analysis steps, then profile with sample binaryninja. Done means the relocatable_regions bottleneck is reduced for the reported segment and section scale without changing analysis results.

Written by the indexing model from the issue text.

Description

Component: WARP File Format: SharedCache

Version and Platform (required):

  • Binary Ninja Version: 5.3.9417-dev Ultimate, 8f260d72
  • OS: macos
  • OS Version: 26.3.1
  • CPU Architecture: arm64

Bug Description:
I've been testing analysis performance with a full iOS shared cache loaded to look for scalability bottlenecks. This has 4,207 images loaded from the shared cache, with 22,222 segments, 132,809 sections, over 15 million functions, and 31 million symbols.

One bottleneck I noticed is in WARP's relocatable_regions:

https://github.com/Vector35/binaryninja-api/blob/24eabc032f6450001bd799b5ec2dc368467b773a/plugins/warp/src/lib.rs#L425-L441

Specifically, this calls BinaryView::GetSectionsAt for each segment in the binary. Since there are many threads running this same activity in parallel, this leads to heavy contention on the lock used internally for the atomic accesses to BinaryView::m_sectionMap.

Steps To Reproduce:

  1. Open a shared cache with an initial analysis hold.
  2. Select all images in the shared cache image list, and hit Load Selected.
  3. Wait the 5-10 minutes for them to load.
  4. Clear the analysis hold.
  5. Once RTTI scanning and disassembly have concluded and we reach phase 1 of analysis, run sample binaryninja to observe where CPU time is being spent.

Additional Information:
Even the call to BinaryViewExt::segments is relatively expensive in this context since the underlying core C API must allocate a ~175KB buffer to hold the array of segment pointers, and then increment each of their ref counts.

Dominant language
C++
Stars
1.3k
Forks
298
Avg merge
5d 5h
Merged PRs (30d)
19

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Vector35/binaryninja-api

All issues in Vector35/binaryninja-api

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.