idalib: `open_database` fails to handle existing database files and exits prematurely
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 726
- Avg merge
- 11d 11h
- Merged PRs (30d)
- 7
Description
### Description
When using `ida` as backend, analyzing any file with an existing database (packed or unpacked) will cause capa to exit with return code 1, without any logs or cleanup.
I tried `ida -R `, and it says `Switch '-R' can be used only when loading a new file`, which I believe is the reason behind failed `open_database` call.
I can confirm the error is introduced in https://github.com/mandiant/capa/commit/0686305f43920275c1c7ad606cacb45b5c373cb5 , which passes switch `-R` to `idapro.open_database`. `idapro.open_database` will directly cause `exit_group(1)` instead of returning an error code.
### Steps to Reproduce
1. Open `tests/data/009c2377b67997b0da1579f4bbc822c1.exe_` in IDA
2. Save database (produce `.i64` file in same directory) and exit IDA
3. Run `capa -b ida tests/data/009c2377b67997b0da1579f4bbc822c1.exe_ -d`
Note, in current version of capa (commit 6579e01d), because of the lack of `idapro.close_database`, running `capa -b ` will cause idapro to leave behind unpacked database files (`*.id0 *.id1 *.id2 *.nam *.til`); When trying to `open_database` against the file again, it will pick up the unpacked databases and cause the same issue.
**Expected behavior:**
capa displays analyze report correctly.
**Actual behavior:**
capa exits with return code 1 after `capa.loader: idalib: opening database... | loader.py:396`. Any code after `idapro.open_database` is not executed.
### Versions
capa: commit 6579e01d
IDA Pro: Version 9.3.260213.91fc47de Linux x86_64 (64-bit address size)
Python: 3.13.5
OS: Arch Linux
### Additional Information
Related issue: #2808
---
I'd be happy to implement the fix once we decide on a solution.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in loader.py around line 396 and inspect the idalib path using idapro.open_database, including the related database lifecycle and cleanup behavior. Reproduce with tests/data/009c2377b67997b0da1579f4bbc822c1.exe_ and an existing .i64 database; done means capa produces its analysis report without premature exit or leftover unpacked database files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- reverse-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100