avast / avast/retdec

Automatically optimize the sequence of passes that are going to be executed

Open
#874 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
8.6k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

Some retdec passes are very heavy, so retdec fails on some pretty small binaries (i.e. ~3 MiB) if/when they are enabled because of OOM.

So, it may be useful to disable them - even a partially decompiled binary is better than nothing.

But we want to keep as much of them as our machine can.

So ... we need the following subsystem
1. estimating resource consumption for each pass based on known data. 2 kinds of consumption for every resource: peak consumption required when pass is being prossessed and amount required to be allocated before the pass and maybe other passes (i.e. when other passes populate some data used in "this" pass).
2. checking if the enough of resource is available
3. estimating overall usefullness of the system of passes
4. constrained optimization of the system of passes for the best estimated utility within the limits of available resources. You will likely need [rbfopt](https://github.com/coin-or/rbfopt) MSRSM optimizer, as it seems it performs best of the ones I have studied. Or maybe some other unconstrained ([I have a lib providing a unified interface to them, and also implementing some features not supported natively by some optimizers](https://github.com/KOLANICH/UniOpt.py)), using resource consumption as a penalty.
5. after each pass the plan should be recalculated, taking into account the real resources consumed by the pass and additional information generated within it.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating RetDec’s pass-execution subsystem and inventorying how passes currently run; the issue names no files or tests. Review the linked rbfopt and UniOpt.py projects, then clarify resource measurements, utility scoring, constrained selection, and when plans are recalculated. Done means pass selection respects available resources while retaining as much estimated usefulness as possible.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.