Vector35 / Vector35/binaryninja-api

Calling `AnalysisContext::SetMediumLevelILFunction` on large/complex functions can take an extremely long time

Open
#7,746 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Component: API Impact: Medium
Dominant language
C++
Stars
1.3k
Forks
298
Avg merge
5d 5h
Merged PRs (30d)
19

Description

Version and Platform (required):

  • Binary Ninja Version: 5.3.8707-dev Ultimate (a74ed668)
  • Edition: Ultimate
  • OS: macOS
  • OS Version: 15.6
  • CPU Architecture: M1

Bug Description:
I have a plugin that has a function workflow activity that makes a copy of MLIL and then calls AnalysisContext::SetMediumLevelILFunction (after Finalize and GenerateSSAForm), that call can take an extremely long time (its been stuck for over an hour). I encountered this issue with the function _curve25519_donna found in the CoreUtils image in the DYLD Shared Cache. It seems MediumLevelILFunction::GetLLILSSAToMLILExprMap is the main issue, spending lots of time looping through entries in m_translationData->mlilToMlilExprMap. In the case of _curve25519_donna it has 28283 entries.

Steps To Reproduce:

  1. Build and install the native plugin shown below.
  2. Open with options a copy of the DYLD Shared Cache for iOS 26.0 for an iPhone 17 Pro Max, and make sure to select the core.function.issue function workflow.
  3. Load the image /System/Library/PrivateFrameworks/CoreUtils.framework/CoreUtils.
  4. Wait a short amount of time.
  5. Observe analysis will appear to be stuck (the progress won't be changing).
  6. In the console execute the following bv.analysis_info and it will probably only show one active info for the function _curve25519_donna.

Binary:
A native plugin that registers a function workflow that triggers the issue:

#include "binaryninjaapi.h"
#include "binaryninjacore.h"
#include "mediumlevelilinstruction.h"

using namespace BinaryNinja;

static void Action(Ref<AnalysisContext> ctx)
{
    const auto func = ctx->GetFunction();

    if (func->GetSymbol()->GetRawName() != "_curve25519_donna")
        return;

    const auto bv = func->GetView();

    const auto mlil = ctx->GetMediumLevelILFunction();
    if (!mlil)
        return;

    const auto ssa = mlil->GetSSAForm();
    if (!ssa)
        return;

    Ref<MediumLevelILFunction> newMlil = new MediumLevelILFunction(mlil->GetArchitecture(), func);
    newMlil->PrepareToCopyFunction(mlil);

    for (const auto& oldBlock : mlil->GetBasicBlocks()) {
        newMlil->PrepareToCopyBlock(oldBlock);

        for (size_t oldInstrIdx = oldBlock->GetStart(), oldBlockEnd = oldBlock->GetEnd(); oldInstrIdx < oldBlockEnd; oldInstrIdx++) {
            const auto oldInstr = mlil->GetInstruction(oldInstrIdx);
            newMlil->SetCurrentAddress(oldBlock->GetArchitecture(), oldInstr.address);
            newMlil->AddInstruction(oldInstr.CopyTo(newMlil), oldInstr);
        }
    }

    newMlil->Finalize();
    newMlil->GenerateSSAForm();
    ctx->SetMediumLevelILFunction(newMlil);
}

extern "C" {

BN_DECLARE_CORE_ABI_VERSION

static constexpr auto FunctionWorkflowInfo = R"#({
    "title": "Analysis Issue",
    "description": "",
    "targetType" : "function"
})#";

BINARYNINJAPLUGIN bool CorePluginInit()
{
    const auto functionWorkflow = Workflow::Instance("core.function.metaAnalysis")->Clone("core.function.issue");
    const auto config = R"#({"name":"core.function.issue.analysis-stall","title":"Causes an analysis stall","description":"","role":"action","eligibility":{"auto": {},"predicates": [{"type": "viewType", "value": ["DSCView","Mach-O"], "operator": "in"}]}})#";
    const auto activity = new Activity(config, &Action);
    functionWorkflow->RegisterActivity(activity);
    functionWorkflow->InsertAfter("core.function.generateMediumLevelIL", "core.function.issue.analysis-stall");
    Workflow::RegisterWorkflow(functionWorkflow, FunctionWorkflowInfo);
    return true;
}
}

Additional Information:
Whilst the call to AnalysisContext::SetMediumLevelILFunction is ongoing Binary Ninja seems to stop doing all other work and is just using a single thread.

The issue can also cause the UI to freeze. In one case I right-clicked on the function whilst the analysis was ongoing and the UI froze. In another case just navigating to the function caused the UI to freeze.

  • In the case of navigating this is caused by Sidekick requesting HLIL when the linear view address changes. Its doing it in the callback and because this issue causes it to take an extremely long time to generate it freezes the UI. Given requesting HLIL might not be an instant operation it might be best to take this code off of the UI thread.
  • I'm not sure whats going on when right-clicking the function _curve25519_donna is causing the UI to freeze but here's the call stack:
__psynch_cvwait (@__psynch_cvwait:5)
_pthread_cond_wait (@_pthread_cond_wait:249)
std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) (@std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&):11)
___lldb_unnamed_symbol15407 (@___lldb_unnamed_symbol15407:20)
___lldb_unnamed_symbol18043 (@___lldb_unnamed_symbol18043:65)
___lldb_unnamed_symbol18041 (@___lldb_unnamed_symbol18041:42)
___lldb_unnamed_symbol18412 (@___lldb_unnamed_symbol18412:28)
LinearView::addressForCall() (@LinearView::addressForCall():73)
___lldb_unnamed_symbol21241 (@___lldb_unnamed_symbol21241:15)
MenuInstance::layoutMenu(std::__1::map<QString, QString, std::__1::less<QString>, std::__1::allocator<std::__1::pair<QString const, QString>>> const&, QString const&, bool) (@MenuInstance::layoutMenu(std::__1::map<QString, QString, std::__1::less<QString>, std::__1::allocator<std::__1::pair<QString const, QString>>> const&, QString const&, bool):410)
MenuInstance::update(UIActionHandler*, UIActionContext const&, bool) (@MenuInstance::update(UIActionHandler*, UIActionContext const&, bool):53)
MenuInstance::update(UIActionHandler*, bool) (@MenuInstance::update(UIActionHandler*, bool):55)
Menu::create(QWidget*, UIActionHandler*, bool) (@Menu::create(QWidget*, UIActionHandler*, bool):42)
ContextMenuManager::show(QPoint, Menu*, UIActionHandler*) (@ContextMenuManager::show(QPoint, Menu*, UIActionHandler*):27)
LinearView::mousePressEvent(QMouseEvent*) (@LinearView::mousePressEvent(QMouseEvent*):247)
QWidget::event(QEvent*) (@QWidget::event(QEvent*):31)
QFrame::event(QEvent*) (@QFrame::event(QEvent*):16)
QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*, QEvent*) (@QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*, QEvent*):67)
QApplicationPrivate::notify_helper(QObject*, QEvent*) (@QApplicationPrivate::notify_helper(QObject*, QEvent*):79)
QApplication::notify(QObject*, QEvent*) (@QApplication::notify(QObject*, QEvent*):1186)
QCoreApplication::sendSpontaneousEvent(QObject*, QEvent*) (@QCoreApplication::sendSpontaneousEvent(QObject*, QEvent*):46)
QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) (@QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool):244)
QWidgetWindow::handleMouseEvent(QMouseEvent*) (@QWidgetWindow::handleMouseEvent(QMouseEvent*):337)
QWidgetWindow::event(QEvent*) (@QWidgetWindow::event(QEvent*):28)
QApplicationPrivate::notify_helper(QObject*, QEvent*) (@QApplicationPrivate::notify_helper(QObject*, QEvent*):87)
QApplication::notify(QObject*, QEvent*) (@QApplication::notify(QObject*, QEvent*):120)
QCoreApplication::sendSpontaneousEvent(QObject*, QEvent*) (@QCoreApplication::sendSpontaneousEvent(QObject*, QEvent*):46)
QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) (@QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*):489)
QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) (@QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>):105)
QCocoaEventDispatcherPrivate::postedEventsSourceCallback(void*) (@QCocoaEventDispatcherPrivate::postedEventsSourceCallback(void*):132)
__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ (@__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__:10)
__CFRunLoopDoSource0 (@__CFRunLoopDoSource0:46)
__CFRunLoopDoSources0 (@__CFRunLoopDoSources0:61)
__CFRunLoopRun (@__CFRunLoopRun:213)
CFRunLoopRunSpecific (@CFRunLoopRunSpecific:146)
RunCurrentEventLoopInMode (@RunCurrentEventLoopInMode:84)
ReceiveNextEventCommon (@ReceiveNextEventCommon:57)
_BlockUntilNextEventMatchingListInModeWithFilter (@_BlockUntilNextEventMatchingListInModeWithFilter:22)
_DPSNextEvent (@_DPSNextEvent:174)
-[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] (@-[NSApplication(NSEventRouting) _nextEventMatchingEventMask:untilDate:inMode:dequeue:]:175)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the native plugin workflow on the _curve25519_donna function and inspect AnalysisContext::SetMediumLevelILFunction, especially MediumLevelILFunction::GetLLILSSAToMLILExprMap and m_translationData->mlilToMlilExprMap. Use bv.analysis_info during the run and verify that analysis completes without freezing other work or the UI.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.