Fallout-build / Fallout-build/Fallout

Expand BenchmarkDotNet coverage to other performance-critical paths

Open
#258 0 comments 0 reactions 0 assignees View on GitHub
enhancement target/backlog
Dominant language
C#
Stars
154
Forks
19
Avg merge
1d 22h
Merged PRs (30d)
15

Description

## Why
Performance matters for a build tool: every consumer's run pays to parse/resolve the world before any real work. As we own more code (e.g. the inlined `vs-solutionpersistence` parser per #248), we should **prove** non-regressions with numbers, not vibes.

Started in the initial slnx benchmark PR (link when filed): `tests/Benchmarks/Fallout.Persistence.Solution.Benchmarks/` covers `.slnx` parsing across 1/10/100/1000 projects × {flat, foldered}.

## Candidates worth measuring
Sorted by likely build-time impact, biggest first. Not committed scope — just things worth costing before someone optimises by gut.
- **Tool wrapper invocation overhead** (`Fallout.Tooling.ToolTasks.Run` / `Configure` round-trips). Every `DotNetTest`/`DotNetBuild`/etc. goes through this. Likely subprocess-dominated, but per-call wrapping cost adds up.
- **Source generator throughput** — `StronglyTypedSolutionGenerator`, `TransitionShimGenerator`. Run at compile time on every build; per-target cost shows up as IDE responsiveness.
- **File globbing** (`Fallout.Utilities.IO.Globbing`). Build code globs heavily ("all .csproj under src/", "all .verified.cs under tests/"). Worth knowing the cost vs .NET's `Directory.EnumerateFiles` + regex.
- **`AbsolutePath` ops** — normalisation, joining, parent/relative. Every framework call touches these. Allocations matter (each `/` makes a new path).
- **MSBuild project parsing** (`Project.GetMSBuildProject`). Goes through `Microsoft.Build.Locator` + real MSBuild evaluation. Heavy. Probably can't be fast, but the absolute cost informs caching.
- **Configuration-attribute → host serialisation** (`SchemaUtility`, YAML/JSON emitters). Runs on `fallout :setup` and CI-config regen, not hot path, but the STJ migration (#114) deserves a check.
- **Tool wrapper code generation** (`Fallout.Tooling.Generator`) — only on `./build.ps1 GenerateTools`, off hot path, but useful if we ever make it incremental.
- **Telemetry write path** — every build calls it once; should be noise-level cheap.

## Out of scope (legacy / known-cheap / no actionable lever)
- `.sln` (legacy format) parser — untouched per #248.
- Anything in `Fallout.Migrate` — runs once per consumer's lifetime.
- Verify snapshot diffing — framework, not our code.

## Suggested shape for new benchmark projects
Mirror what we just did:
- Live under `tests/Benchmarks/Fallout..Benchmarks/`.
- BenchmarkDotNet via `Directory.Packages.props`, no per-project package add.
- Compile-only in CI; local via `dotnet run --project ... -c Release`.
- Fixtures generated programmatically in `[GlobalSetup]` — deterministic, no committed binary blobs.
- README per project: what it measures, why, how to invoke.

## Done when
Tracking issue, not a single deliverable. Each candidate could be its own PR. Close when the top 3 (tool wrapper, source generators, globbing) have coverage.

## Coordinates with
- The initial `Fallout.Persistence.Solution.Benchmarks` work (separate PR filed alongside).
- Any future "is X fast enough" discussion — answer with numbers from these.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.