Nimblesite / Nimblesite/SharpLsp

Flaky e2e: profiler object-graph test fails when the baseline heap dump has no StringBuilder instances

Open
#196 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
TypeScript
Stars
132
Forks
5
Avg merge
6h 24m
Merged PRs (30d)
27

Description

Summary

e2e_modules::profiler_dump_analysis_full_stack::test_profiler_object_graph_roots_inspect_and_diff_full_stack fails intermittently on the assertion at tests/e2e_modules/profiler_dump_analysis_full_stack.rs:117:

baseline heap dump must contain StringBuilder instances (ProfileTarget allocates them constantly)

Observed 1 failure in 3 consecutive full runs of Rust shard 1 on the same commit and machine (Windows, make _test-rust-shard SHARD=1 SHARD_COUNT=2). The failing run reached the assertion in 4.86s; the passing runs took the same path. Nothing in the tree changed between runs.

Why it flakes

The assertion depends on ProfileTarget having actually allocated StringBuilders before the baseline dump is captured. The comment ("allocates them constantly") is true in steady state, but there is a race between the target reaching that steady state and dotnet-dump capturing the heap. On a loaded machine the dump can land in the startup window, where the assertion is legitimately false.

A distinct, earlier failure mode of the same test is a missing dotnet-dump global tool — that produces a much faster failure (~0.4s) and is an environment problem, not this race. CI installs dotnet-trace, dotnet-counters, and dotnet-dump explicitly in ci-rust.yml, so only the race applies there.

Suggested fix

Do not assert on an unsynchronised allocation. Either:

  • have ProfileTarget signal readiness (stdout marker / file touch) once it has allocated the objects the assertion needs, and capture the baseline only after that, or
  • poll: capture and inspect until StringBuilder appears or a timeout elapses, so the assertion reflects steady state rather than whenever the dump happened to land.

Per the repo's test rules the assertion must not be weakened or removed — it should be made deterministic.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at tests/e2e_modules/profiler_dump_analysis_full_stack.rs:117 and reproduce with make _test-rust-shard SHARD=1 SHARD_COUNT=2. Trace when ProfileTarget allocates StringBuilder instances relative to the baseline dotnet-dump capture, then make the existing assertion deterministic without weakening it. Done means the test waits for the required objects and no longer intermittently fails during the startup window.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.