dotnet / dotnet/efcore

Shaper has excessive calls to DataReader and ValueConverters around ternary projections

Open
#37,109 6 comments 0 reactions 1 assignee Claimed by @AndriySvyryd View on GitHub
area-json area-perf area-query customer-reported
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

### Bug description

When generating a shaper for a query, it appears each property assignment is evaluated individually and read into a local variable.

This includes values that may not be used in the final projection in cases where the projection contains a ternary.

For example if I have a ternary where in one case I use a single field, and in a second case I use that field, and an additional 2 fields, the generated shaper will look something like

variable1 = dataReader.GetInt(0)
variable2 = dataReader.GetInt(0)
variable3= dataReader.GetInt(1)
variable4 = dataReader.GetInt(2)

? new { variable1 } : new {variable2, variable3, variable4 }

This doesn't seem that terrible, but when combined with ValueConverters it can be absolutely horrendous.

The attached example doesn't show it, just the core issue. by my actual case was that I have type heirarachy with fields and a single JSON "additional data" field. This is a schema that was not generated by EF, and that I cannot change.

What I did was to add a ValueConverter that deserialized the JSON into a field on derived types. This worked, until we queries about 300 records and it was taking over a minute, even though the query took 1/10 of a second.

Digging in, what was happening was that if I projected an object from the deserialized JSON, the shaper was generating locals _for each property assigned from it_.

Someing like:

variable1 = for field 0
variable2 = for field 0
variable3 = for field 0

return new { variable1.PropertyOne, variable2.PropertyTwo, variable3.Property3

In this case deserializing three times.

Combined with loading data for all possible outputs prior to seeing which type is actually going to be emitted turned into about 150 deserializations of this object _per row_. (About 12 possible types with an average 10 or so fields each).

### Your code

```csharp
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;

var context = new TestDbContext();

// This generates a query that casts id to a string, and a shaper that will
// 1. Read the id field twice into two different local variables
// 2. _Always_ read the casted string field, even if the result is not used.
await context.Items.Select( i => i is Item1? new ItemProjection { Id = i.Id, Value = ((Item1) i).SomeProperty, Description = i.Id.ToString()} : i is Item2 ? new ItemProjection() { Id = i.Id, Value = ((Item2)i).SomeOtherProperty } : null).ToListAsync();

// This is just a trivial example showing that the field will be read twice, if used twice in the projection
await context.Items.Select( i => new { Id = i.Id, AlternateID = i.Id} ).ToListAsync();

public class TestDbContext: DbContext
{
public DbSet Items { get; set; } = null!;

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseSqlServer("Server=(localdb)");
}

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.ApplyConfigurationsFromAssembly(typeof(Program).Assembly);
}
}

public class ItemConfiguration: IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.HasDiscriminator(i => i.Type)
.HasValue(ItemType.One)
.HasValue(ItemType.Two);
}
}

public enum ItemType
{
One,
Two,
}

public abstract class Item {
public ItemType Type { get; set; }
public int Id { get; set; }
public int Description { get; set; }
}

class Item1:Item
{
public int SomeProperty { get; set; }
}

class Item2:Item
{
public int SomeOtherProperty { get; set; }
}

public class ItemProjection
{
public int Id { get; set; }
public int Value { get; set; }
public string? Description { get; init; }
}
```

### Stack traces

```text

```

### Verbose output

```text

```

### EF Core version

9.0.10

### Database provider

_No response_

### Target framework

_No response_

### Operating system

_No response_

### IDE

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.