dotnet / dotnet/roslyn

Introduce Attach operator for source generator pipeline triggering incremental output driven by only one source (instead of either like Combine)

Open
#79,322 2 comments 2 reactions 1 assignee Claimed by @333fred View on GitHub
Area-Compilers Concept-API Feature Request
Dominant language
C#
Stars
20.7k
Forks
4.3k
PR merge metrics
PR metrics pending

Description

## Background and Motivation

In the Roslyn incremental generator pipeline, there is currently no built-in mechanism to combine two value providers such that only one side—the primary or "driver" input—triggers recomputation. The existing `Combine` operator causes the downstream pipeline to re-evaluate whenever *either* side changes. While this behaviour is appropriate in general, it introduces inefficiencies in common generator workflows.

An optimal generator needs to:

1. Determine as cheaply as possible list of "inputs" that determine whether generation should occur. This can often be done via syntax analysis only without semantic analysis.
2. Later enrich those inputs with additional contextual data—often requiring semantic model access—to produce the final output.

Due to the design of the current API, developers are forced to perform all semantic enrichment during the `transform` step of `CreateSyntaxProvider`, because that is the only stage where `SemanticModel` is available. Attempting `Combine` with `context.CompilationProvider` will lead to complete loss of incremental generation since compilation changes on every keystroke. As such, all semantic computation (even that which is only relevant for final source rendering vs equivalency checks) needs to happen inside `transform` delegate. As such the object coming out of `CreateSyntaxProvider` must be "cache friendly". Critically, developers must avoid passing `ISymbol` instances through `IncrementalValuesProvider` pipelines. Doing so causes the previous `Compilation` to be rooted in the incremental cache, leading to expensive and unnecessary cache invalidations. `SyntaxNode` can be passed downstream safely *only* if either:

- It is excluded from equality comparison logic, or
- A custom `IEqualityComparer` is used to ignore its reference identity (which is not value-comparable by default).

This leads to developers having to either having to create bulky model objects that must preemptively capture all possibly needed semantic data—even for unchanged inputs—just to preserve cache correctness, or generating final source output right in template. Alternatively they must compute the final string output right inside `transform` method and use that as the cache key. In both instances they will need to query semantic model which may not be necessary for minimal set of data that forms cache key. This undermines the purpose of a multi-step pipeline and can result in redundant semantic analysis.

The proposed `Attach` API enables a safe and efficient pattern: combine key-driven model tied to actual "incremental value change" (such as those coming from `CreateSyntaxProvider`) with a context value (such as `Compilation` snapshot obtained from `context.CompilationProvider`), in a way that guarantees recomputation only when "one side" changes, not "either" like in `Combine`. This permits semantic enrichment to occur *after* determining that a generation is needed, minimizing both semantic model overhead and cache churn. Even in scenarios where semantic information may need to be resolved twice (first in transform and then in Attach), if we can avoid accessing semantic model entirely due to cache hits driven by pure syntax model, it is likely a worthy tradeoff.

## Proposed API

```diff
namespace Microsoft.CodeAnalysis;

public static class IncrementalValueProviderExtensions
{
+ public static IncrementalValueProvider Attach(
+ this IncrementalValuesProvider left,
+ IncrementalValueProvider right,
+ Func transform)
+ where TLeft : notnull;
}
```

This method behaves similarly to `Combine`, but only propagates changes downstream when the **left** input changes. The **right** input is available for enrichment or context purposes (e.g., compilation or semantic model) but does not trigger recomputation on its own.

## Usage Examples

``` C#
var syntaxModels = context.SyntaxProvider
.CreateSyntaxProvider(
predicate: static (node, _) => node.IsPartialTypeDeclaration(),
transform: static (context, _) => new ReducedModel(context.Node))
.WithComparer(ReducedModelKeyComparer.Instance); // Node is carried, but not part of the comparison.

var renderingView = syntaxModels.Attach(context.CompilationProvider, static (generationModel, compilation) =>
{

// we only arrive here if we determined that
// Compilation is used to enrich the model, but doesn't trigger incremental change on its own

var semanticModel = compilation.GetSemanticModel(generationModel.Node);
return RenderModel(generationModel, semanticModel); // this is our source "view" that can use our custom model, node in it and semantic model to generate rendered source string
});

context.RegisterSourceOutput(renderingView, static (ctx, data) =>
{
var source = SourceRenderer.Render(data);
ctx.AddSource(data.FileNameHint, source);
});
```

## Alternative Designs

Overload to `Combine` that takes `propagateOnlyIfLeftChanged: bool` flag

## Risks

- Introducing a new propagation model might complicate mental models of the incremental pipeline, especially for less experienced users.
- In some scenarios can lead to degradation of performance. This can happen when semantic model is resolved in `transform` and selection criteria for caching doesn't trigger often enough to be worth the creating SemanticModel twice (first in transform and later in pipeline). This can largely be mitigated through developer education to select optimal strategy.
- No compatibility issues are expected, as this is a purely additive API.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.