mandiant / mandiant/capa

idalib: `open_database` fails to handle existing database files and exits prematurely

Open
#2,950 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug ida
Dominant language
Python
Stars
6.2k
Forks
726
Avg merge
11d 11h
Merged PRs (30d)
7

Description

### Description

When using `ida` as backend, analyzing any file with an existing database (packed or unpacked) will cause capa to exit with return code 1, without any logs or cleanup.

I tried `ida -R `, and it says `Switch '-R' can be used only when loading a new file`, which I believe is the reason behind failed `open_database` call.
I can confirm the error is introduced in https://github.com/mandiant/capa/commit/0686305f43920275c1c7ad606cacb45b5c373cb5 , which passes switch `-R` to `idapro.open_database`. `idapro.open_database` will directly cause `exit_group(1)` instead of returning an error code.

### Steps to Reproduce

1. Open `tests/data/009c2377b67997b0da1579f4bbc822c1.exe_` in IDA
2. Save database (produce `.i64` file in same directory) and exit IDA
3. Run `capa -b ida tests/data/009c2377b67997b0da1579f4bbc822c1.exe_ -d`

Note, in current version of capa (commit 6579e01d), because of the lack of `idapro.close_database`, running `capa -b ` will cause idapro to leave behind unpacked database files (`*.id0 *.id1 *.id2 *.nam *.til`); When trying to `open_database` against the file again, it will pick up the unpacked databases and cause the same issue.

**Expected behavior:**

capa displays analyze report correctly.

**Actual behavior:**

capa exits with return code 1 after `capa.loader: idalib: opening database... | loader.py:396`. Any code after `idapro.open_database` is not executed.

### Versions

capa: commit 6579e01d
IDA Pro: Version 9.3.260213.91fc47de Linux x86_64 (64-bit address size)
Python: 3.13.5
OS: Arch Linux

### Additional Information

Related issue: #2808

---

I'd be happy to implement the fix once we decide on a solution.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in loader.py around line 396 and inspect the idalib path using idapro.open_database, including the related database lifecycle and cleanup behavior. Reproduce with tests/data/009c2377b67997b0da1579f4bbc822c1.exe_ and an existing .i64 database; done means capa produces its analysis report without premature exit or leftover unpacked database files.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.