microsoft / microsoft/pyright

Consuming large amount of heap memory

Open
#10,881 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug question
Dominant language
Python
Stars
15.6k
Forks
1.8k
Avg merge
12h 13m
Merged PRs (30d)
52

Description

Our project is consuming over 2GB of memory, running out of memory in CI without bumping the default node limit, like this: NODE_OPTIONS: "--max-old-space-size=6144".

--verbose shows: Heap stats: total_memory_size=36864MB, total_free_size=73MB, total_heap_size=2427MB, used_heap_size=2315MB, cross_worker_used_heap_size=2315MB, total_physical_size=2424MB, total_available_size=1776MB, heap_size_limit=4144MB

Full output: https://gist.github.com/xixixao/3a4355735b5f4218a6f07d68ab8c8b2c (it's 100K+ lines - is that expected / desirable? - it's from an older version, but latest 1.1.404 gives the same stats)

Stats:

Poe => pyright -p /foo/pyproject.toml --stats
Loading pyproject.toml file at /foo/pyproject.toml
No include entries specified; assuming /foo
Auto-excluding **/node_modules
Auto-excluding **/__pycache__
Auto-excluding **/.*
Found 918 source files
pyright 1.1.402
0 errors, 0 warnings, 0 informations 
Completed in 32sec

Analysis stats
Total files parsed and bound: 3826
Total files checked: 918

Timing stats
Find Source Files:    0.07sec
Read Source Files:    15.38sec
Tokenize:             1.72sec
Parse:                1.48sec
Resolve Imports:      1.68sec
Bind:                 2.36sec
Check:                9.86sec
Detect Cycles:        0sec

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing pyright -p /foo/pyproject.toml --stats against the reported 918 source files and capture the heap statistics shown by --verbose. Review the linked full output and compare the current results with pyright 1.1.402 and 1.1.404; the issue is complete when the memory behavior is explained and an actionable fix or confirmed expected behavior is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, python
Domain
performance, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.