General Tag Parser memory issues, can I just disable it entirely?

オープン

#7,329 2021年4月9日に作成コメント 7 件リアクション 0 件担当者 0 名 GitHub で見る

まだ誰も着手していません。

bug investigate: repro Language Service performance

主要言語: TypeScript
スター: 6.2k
フォーク: 1.7k
平均マージ: 14時間 46分
マージ済み PR（30日）: 61

説明

Type: LanguageService

Describe the bug

OS and Version: Kubuntu 20.04.2
VS Code Version: 1.55.1
C/C++ Extension Version: 1.2.2
Other extensions you installed (and if the issue persists after disabling them): GitLens, Docker, DeviceTree, Remote - Containers, Remote - SSH (did not disable any of these while testing)
Does this issue involve using SSH remote to run the extension on a remote machine?:
I originally ran remote in a docker container, but have since only access the files outside of the container and still experience the issue.
A clear and concise description of what the bug is, including information about the workspace (i.e. is the workspace a single project or multiple projects, size of the project, etc).
I have been having issues where my host system's memory usage slowly climbs until consuming all RAM and swap memory (that would be 64GB total). It is almost entirely the cpptools process, and if I don't catch it in time and kill it, it will lock up my system. The code base is a large linux system that supports multiple platforms and targets, with several kernel versions available depending on the target. I use this tool to build a compile_commands.json based on the build output, as my work is mostly kernel/driver layer. The issue seems primarily related to when the Tag Parser is in use (I have tried only using the Tag Parser, which was the worst, but now I seem to only observe it when IntelliSense is failing for some reason). I thought I had resolved it by only using IntelliSense with Tag Parser backup disabled, but when I opened this morning it informed me IntelliSense was unavailable (?) and it is using the Tag Parser (the memory usage is not as high yet, but is currently at 10.5GB and keeping fairly steady). Since starting to write this bug, the IntelliSense engine seems to have come back online and it is staying stable (although still at high memory usage seemingly mostly consumed by the Tag Parser).

Steps to reproduce

Open large workspace (project described above)
Either enable ONLY the Tag Parser, or encounter a situation where the "IntelliSense client is not available"
Observe memory usage increase steadily
Going to the definition of enough files seems to also make it worse (presumably spawning more parsing processes)

Expected behavior

Some amount of memory usage is understood to be necessary, even in the several GB. I would expect after using a certain amount of memory, it limits itself, even if that limits what features are available until it completes.
Most importantly, I expect the setting to explicitly NOT use the Tag Parser as a backup would prevent the Tag Parser from running when IntelliSense is not available. I would rather have nothing than eventual system lock up, although I'm unclear why the IntelliSense client would not be available in the first place (as reported by the C/C++ Output window when I have Debug set).

NOTE: any logs posted have been trimmed of any potential company IP. I included notes to describe what was there and seemed important from my perspective. If more information is necessary, we can try to figure out what I can/can't share there (might not even be necessary, just doing it out of an abundance of caution unless other information is actually useful).

Logs

-------- Diagnostics - 4/9/2021, 10:11:45 AM
Version: 1.2.2
Current Configuration:
{
    "name": "Linux",
    "includePath": [
        "${workspaceRoot}/kmod/**",
        "${workspaceRoot}/software/libs",
        "${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0",
        "${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0/x86_64-linux-gnu",
        "${workspaceRoot}/toolchain/lib/gcc/x86_64-linux-gnu/7.5.0/include",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/include",
        "${workspaceRoot}/kernel/linux-4.19.25-rt16/include",
        "${workspaceRoot}/x86_64-linux-gnu/intevac/libs",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/dbus-1.0",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/dbus-1.0/include",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/libxml++-3.0",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glibmm-2.4",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glibmm-2.4/include/",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glib-2.0",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/libxml++-3.0/include/",
        "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glib-2.0/include/"
    ],
    "defines": [
        "QSC_CONFIG_AUDIO_ENGINE"
    ],
    "intelliSenseMode": "linux-gcc-x64",
    "compilerPath": "/usr/bin/g++",
    "cStandard": "c11",
    "cppStandard": "c++17",
    "compileCommands": "${workspaceFolder}/compile_commands.json",
    "compilerArgs": [],
    "intelliSenseModeIsExplicit": true,
    "cStandardIsExplicit": true,
    "cppStandardIsExplicit": true,
    "compilerPathIsExplicit": true,
    "browse": {
        "path": [
            "${workspaceRoot}/kmod/**",
            "${workspaceRoot}/software/libs",
            "${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0",
            "${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0/x86_64-linux-gnu",
            "${workspaceRoot}/toolchain/lib/gcc/x86_64-linux-gnu/7.5.0/include",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/include",
            "${workspaceRoot}/kernel/linux-4.19.25-rt16/include",
            "${workspaceRoot}/x86_64-linux-gnu/intevac/libs",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/dbus-1.0",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/dbus-1.0/include",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/libxml++-3.0",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glibmm-2.4",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glibmm-2.4/include/",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glib-2.0",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/libxml++-3.0/include/",
            "${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glib-2.0/include/",
            "${workspaceFolder}"
        ],
        "limitSymbolsToIncludedHeaders": true
    }
}
Translation Unit Mappings:
[ OMITTED PATH TO SINGLE IP SOURCE (C FILE) ]:
    OMITTED PATH TO SINGLE IP SOURCE (C FILE)
    OMITTED PATH TO SINGLE IP SOURCE (H FILE)
Translation Unit Configurations:
[ OMITTED PATH TO SINGLE IP SOURCE (C FILE) ]:
    Process ID: 1514214
    Memory Usage: 84 MB
    Compiler Path: /usr/bin/g++
    Includes:
    
NOTE: INCLUDE PATHS TRIMMED TO HIDE COMPANY IP
-- there were 35 include paths in this list, spanning our IP as well as source for a single kernel version

        /usr/lib/gcc/x86_64-linux-gnu/9/include
        /usr/local/include
        /usr/include/x86_64-linux-gnu
        /usr/include
    Defines:
        __KERNEL__
        KASAN_SHADOW_SCALE_SHIFT=3
        KASAN_SHADOW_SCALE_SHIFT=3
        MODULE
        KBUILD_BASENAME="output"
        KBUILD_MODNAME="vst_controller"
    Forced Includes:
    
NOTE: INCLUDE PATHS TRIMMED TO HIDE COMPANY IP
-- there were 3 include paths in this list, spanning our IP as well as source for a single kernel version

    Standard Version: c11
    IntelliSense Mode: linux-gcc-x64
    Other Flags:
        --gcc
        --gnu_version=90300
    compile_commands.json entry:
        directory: /home/qsysbuilder/qsys-develop
        file: <omitted>
        
NOTE: FILE BUILD OPTIONS TRIMMED TO HIDE COMPANY IP
-- it was used to pull the include paths 

Total Memory Usage: 84 MB
Browse Paths from compile_commands.json, from workspace folder: /home/james/qsys-develop

NOTE: BROWSE PATHS TRIMMED TO HIDE COMPANY IP
-- there were 1612 include paths in this list, spanning our IP as well as source for several kernel versions

------- Workspace parsing diagnostics -------
Number of folders and files enumerated: 1108129
Number of files discovered (not excluded): 429872
Number of files parsed: 39641

Screenshots

Additional context

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
リポジトリをフォークし、ブランチを切って変更します。
issue 番号を参照したプルリクエストを送ります。

調査の方向性

まず、compile_commands.json 構成と C/C++ Output ウィンドウを使用して、説明されている大規模なワークスペースで Tag Parser と IntelliSense fallback によるメモリ増加を再現します。Tag Parser backup を無効にした場合の挙動を比較し、IntelliSense が利用できない場合に設定が尊重されるかどうかを判断します。完了条件は、失敗モードと、メモリを上限内に抑える、または無効化する挙動が、焦点を絞った regression test でカバーされることです。

索引モデルが issue の本文から書いたものです。

評価

技術スタック: cpp
領域: devtools
issue の種類: バグ
難易度: 4/5
見積もり時間: 3〜5日
活発さ: 停滞
明瞭さ: 説明が足りない
初心者へのやさしさ: 35/100

General Tag Parser memory issues, can I just disable it entirely?

説明

コントリビューションガイド

はじめの一歩

調査の方向性

評価

新しい issue をメールで受け取る