dotnet / dotnet/efcore

EF 11 regression: QuerySqlGenerator now throws for third-party SqlExpressions

Open
#38,977 0 comments 1 reaction 0 assignees View on GitHub
customer-reported
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

### Bug description

Before efcore 11, we could define our own `SqlExpression` nodes for SQL that EF has no built-in node for and have it render on **any** provider, because `QuerySqlGenerator`'s `VisitExtension` fell back to `base.VisitExtension` → `VisitChildren` for unknown nodes. The node could "self-render" by feeding SQL fragments and child expressions back through the visitor.

EF Core 11 removed that fallback in #37533 ("Remove SqlExpressionVisitor") — the default arm of `QuerySqlGenerator.VisitExtension` now throws InvalidOperationException on an unmapped expression.

This is a regression and a breaking change for which there is no clear upgrade path for provider agnostic libraries.

Possible workarounds:

- Provider specific subclasses of the sql generator.
- Toy wiht `SqlFunctionExpression` and use a `DbCommandInterceptor` to rewrite sql text (🙊)

**Ask:** either restore the `VisitChildren` fallback for unknown nodes, or add an opt-in extension point (e.g. an interface with `GenerateSql(IRelationalCommandBuilder sql, Func visit)` handled in the default arm) — the latter keeps #37533's goal of throwing loudly for genuinely unhandled nodes while giving extensions a supported path.

Happy to contribute a PR if we can settle on a approach

### Your code

```csharp
#:package Microsoft.EntityFrameworkCore.Sqlite@10.0.5
//#:package Microsoft.EntityFrameworkCore.Sqlite@11.0.0-rc.1.26425.128
#:property PublishAot=false
// Run with `dotnet run repro.cs` (.NET 11 RC SDK; the EF 11 package targets net11.0).
//
// On EF Core 10 (swap the package line for @10.0.5), this prints:
// SELECT "o"."Id", RANK() OVER (ORDER BY "o"."Price") AS "Rank"
// FROM "Orders" AS "o"
//
// On EF Core 11.0.0-rc.1, it throws:
// System.InvalidOperationException: Unhandled expression '[RankExpression]' of type
// 'RankExpression' encountered in 'QuerySqlGenerator'.
//
// Cause: dotnet/efcore#37533 removed the VisitExtension -> VisitChildren fallback that
// let third-party SqlExpression nodes render themselves on any relational provider.

using System.Linq.Expressions;
using System.Reflection;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Diagnostics;
using Microsoft.EntityFrameworkCore.Query;
using Microsoft.EntityFrameworkCore.Query.SqlExpressions;

using var db = new AppDb();
var query = db.Orders.Select(o => new { o.Id, Rank = Db.Rank(o.Price) });
Console.WriteLine(query.ToQueryString());

// Server-side marker translated below; never runs on the client.
public static class Db
{
public static double Rank(double orderBy) => throw new NotSupportedException();
}

public class Order
{
public int Id { get; set; }
public double Price { get; set; }
}

public class AppDb : DbContext
{
public DbSet Orders => Set();

protected override void OnConfiguring(DbContextOptionsBuilder options) => options
.UseSqlite("Data Source=:memory:")
.ReplaceService()
.ReplaceService();
}

// Nullability processing has a documented hook for custom SqlExpressions
// (SqlNullabilityProcessor.VisitCustomSqlExpression) — SQL generation no longer has any.
public class ProcessorFactory(RelationalParameterBasedSqlProcessorDependencies dependencies)
: IRelationalParameterBasedSqlProcessorFactory
{
public RelationalParameterBasedSqlProcessor Create(RelationalParameterBasedSqlProcessorParameters parameters)
=> new Processor(dependencies, parameters);
}

public class Processor(
RelationalParameterBasedSqlProcessorDependencies dependencies,
RelationalParameterBasedSqlProcessorParameters parameters)
: RelationalParameterBasedSqlProcessor(dependencies, parameters)
{
protected override Expression ProcessSqlNullability(Expression queryExpression, ParametersCacheDecorator decorator)
=> new NullabilityProcessor(Dependencies, Parameters).Process(queryExpression, decorator);
}

public class NullabilityProcessor(
RelationalParameterBasedSqlProcessorDependencies dependencies,
RelationalParameterBasedSqlProcessorParameters parameters)
: SqlNullabilityProcessor(dependencies, parameters)
{
protected override SqlExpression VisitCustomSqlExpression(
SqlExpression sqlExpression, bool allowOptimizedExpansion, out bool nullable)
{
nullable = true;
return sqlExpression;
}
}

public class TranslatorProvider : RelationalMethodCallTranslatorProvider
{
public TranslatorProvider(RelationalMethodCallTranslatorProviderDependencies dependencies)
: base(dependencies)
=> AddTranslators([new RankTranslator()]);
}

public class RankTranslator : IMethodCallTranslator
{
public SqlExpression? Translate(
SqlExpression? instance,
MethodInfo method,
IReadOnlyList arguments,
IDiagnosticsLogger logger)
=> method.DeclaringType == typeof(Db) && method.Name == nameof(Db.Rank)
? new RankExpression(arguments[0])
: null;
}

// The provider-agnostic "self-rendering" pattern: interleave SQL fragments with real child
// nodes so columns, parameters, aliasing and quoting are still handled by the provider's
// QuerySqlGenerator. Worked on EF Core 8/9/10; EF Core 11 throws before VisitChildren runs.
public class RankExpression(SqlExpression orderBy)
: SqlExpression(typeof(double), orderBy.TypeMapping)
{
public SqlExpression OrderBy { get; } = orderBy;

protected override Expression VisitChildren(ExpressionVisitor visitor)
{
visitor.Visit(new SqlFragmentExpression("RANK() OVER (ORDER BY "));
visitor.Visit(OrderBy);
visitor.Visit(new SqlFragmentExpression(")"));
return this;
}

public override Expression Quote() => throw new NotSupportedException();

protected override void Print(ExpressionPrinter printer) => printer.Append("RANK() OVER (...)");
}
```

### Stack traces

```text

```

### Verbose output

```text

```

### EF Core version

11.0.0-rc.1.26425.128

### Database provider

_No response_

### Target framework

_No response_

### Operating system

_No response_

### IDE

_No response_

Contributor guide

Open the contributing guide

Research direction

Read QuerySqlGenerator.VisitExtension and run the supplied repro.cs against EF Core 10 and 11 to confirm the regression. Done means third-party SqlExpression nodes have a supported provider-agnostic path while genuinely unhandled nodes still fail loudly, with regression coverage for the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.