microsoft / microsoft/terminal

Reduce virtual function usage to mitigate performance penalty due to CFG

Open
#10,610 8 comments 0 reactions 0 assignees View on GitHub
Area-Performance Issue-Feature Product-Conpty Product-Terminal
Dominant language
C++
Stars
105k
Forks
9.6k
Avg merge
3d 17h
Merged PRs (30d)
29

Description

### What is CFG?

[Control Flow Guard (CFG)](https://docs.microsoft.com/en-us/windows/win32/secbp/control-flow-guard) is MSVC/Win32 feature that can combat memory exploits & vulnerabilities. I found this accidentally when I was exploring the performance bottleneck in #10564. I found that even if I replace the entire `GetPatternId` with `return 0`, that call is still very expensive. Then I realized it's not about what's inside the call, it's about the call *itself*. For every virtual function call, the compiler will insert the following instruction before the call:

```asm
call __guard_dispatch_icall_fptr, [SomeMethod]
```

This will then leads to several extra instructions before calling the actual method.

### Disabling CFG

CFG can be disabled in the system setting (restart required). With CFG disabled system-wide, the `call __guard_dispatch_icall_fptr` will simple return quickly without the extra instructions.

CFG can also be disabled in compiler option `(/guard:cf)`. I failed to actually disable this in WT for some reason. I suppose with the option disabled, no `__guard_dispatch_icall_fptr` will be inserted any more.

### Proposed solution

OK here's my refurbished plan;

1. Remove all render related methods in `IRenderEngine` and replace them with a single method `PaintFrame(IRenderData *pData)` (#10615)
1. Replace `InvalidateSomething` methods in `IRenderEngine` with `TriggerSomething` to give the actual engine more control & reduce the coupling with `renderer.cpp`.
1. Let's see what we can do for the left methods.

CC @DHowett @miniksa @zadjii-msft @lhecker for discussion.

Contributor guide

Open the contributing guide

Research direction

Start by reading the IRenderEngine interface and its call sites in renderer.cpp, then review the related PaintFrame proposal in #10615. The issue suggests consolidating render methods, replacing InvalidateSomething methods with TriggerSomething, and assessing the remaining methods. Done would require an agreed design and validated reduction in virtual-call overhead.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.