mandiant / mandiant/capa

optimize rendering of results by avoiding console.print

Open
#2,559 6 comments 0 reactions 0 assignees View on GitHub
performance
Dominant language
Python
Stars
6.2k
Forks
726
Avg merge
11d 11h
Merged PRs (30d)
7

Description

using [line-profiler](https://pypi.org/project/line-profiler/), I found that functions like `render_feature` can be fairly expensive, taking many seconds (cumulatively) to emit their results to the terminal. Digging into this further, it seems that rich's `console.print()` is relatively slow, taking around 100x longer than intermediate string constructions. This means that when we do a lot of little `console.print` calls on parts of a line then performance is poor.

We can optimize this by constructing complete regions or lines up front, and then flushing to the terminal with `console.print`. I think that rich's `Text.append` is still not *very* fast, but its better than doing a terminal write.

Originally we used a StringIO-based strategy of building a large output document and then flushing it in one go (rich-unaware). We might want to migrate back in this direction a little bit.

Contributor guide

Open the contributing guide

Research direction

Start by locating render_feature and the other rendering paths that make many small console.print calls. Use line-profiler to compare terminal output with intermediate string construction, then inspect how Rich Text.append and the earlier StringIO-based strategy could apply. Done means rendering emits larger complete regions or lines with fewer terminal writes while preserving the output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.