dotnet / dotnet/efcore

Unexpected behavior of value-converted expressions in queries

Open
#35,515 4 comments 0 reactions 0 assignees View on GitHub
area-query customer-reported needs-design
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

### Bug description

Using value converters seems to be very error prone, as they can be freely used within LINQ expressions, but in several cases the resulting SQL query does not take value conversion into account.

Apparently (under the assumption that the value conversion is a pure function and that the two directions are indeed inverse), equality comparison between a value-converted column against a constant or parameter matches the "obvious" behavior (the one you would get through client-side evaluation).

Most other expressions seem to be prone to mismatching translations, which can lead to exceptions and/or silently return "wrong" results.

This is likely related to
- https://github.com/dotnet/efcore/issues/10434
- https://github.com/dotnet/efcore/issues/10861
- https://github.com/dotnet/efcore/issues/11347

IIUC the plan was to add a warning (see the comment https://github.com/dotnet/efcore/issues/11347#issuecomment-375164887) but I was unable to trigger it when running the code and I did not it in the documentation.

If there is still a desire to work towards emitting warnings in these cases, this code example might be a good starting point for further testing/bug hunting (I did not find a PR or an issue with this specific goal).

NOTE: I am not sure whether the code I have written is valid or not 😅
I might be misunderstanding the explanation of the [limitations](https://learn.microsoft.com/en-us/ef/core/modeling/value-conversions?tabs=data-annotations#limitations) of value converters. The closest one I see is "It isn't possible to query into value-converted properties, e.g. reference members on the value-converted .NET type in your LINQ queries.", but it seems to be aimed specifically at member access, while the issue I am hitting involves simple scalar values (or, rather, expressions on simple scalar values).

If this is not tackled as a bug in EFCore, I think it might be a good idea to extend the documentation to explicitly mention the pitfalls around queries that rely on value-converted fields.

To reproduce, the Sqlite version of the code runs on https://dotnetfiddle.net/ just fine; the SqlServer version needs a DB to connect to.

### Your code

```csharp
// @nuget: Microsoft.EntityFrameworkCore.Sqlite -Version 9.0.1

using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Microsoft.EntityFrameworkCore;

using var db = new BloggingContext();

db.Database.EnsureDeleted();
db.Database.EnsureCreated();

var clientSideEntities = db.MyEntities.ToList().AsQueryable();

// constants work as expected
LogResults(q => q.Where(x => x.MyHex == 16)); // 1(16, 10), 2(16, 16)
LogResults(q => q.Where(x => x.MyDec == 16)); // 2(16, 16)

var myInt = 16;
// parameters work as expected
LogResults(q => q.Where(x => x.MyHex == myInt)); // 1(16, 10), 2(16, 16)
LogResults(q => q.Where(x => x.MyDec == myInt)); // 2(16, 16)

// column-column comparison does not work as expected
LogResults(q => q.Where(x => x.MyHex == x.MyDec));
// expected: 2(16, 16)
// got: 1(16, 10)

// expressions can succeed with unexpected results
LogResults(q => q.Select(x => x.MyHex + 0));
// expected: 16, 16
// got(SqlServer): 256, 256
// got(Sqlite): 10, 10

// expressions can fail on SQL server
LogResults(q => q.Select(x => x.MyHex - 1));
// expected: 15, 15
// got(SqlServer): The data types nvarchar(max) and nvarchar are incompatible in the subtract operator.
// got(Sqlite): 9, 9

void LogResults(Func, IEnumerable> list)
{
var expected = list(clientSideEntities).ToList();
var actual = list(db.MyEntities).ToList();
PrintList("expected: ", expected);
PrintList("got: ", actual);
}

void PrintList(string prefix, IEnumerable list)
{
try
{
Console.WriteLine(prefix + string.Join(", ", list.Select(x => x?.ToString())));
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}

public class BloggingContext : DbContext
{
public DbSet MyEntities { get; set; }

protected override void OnConfiguring(DbContextOptionsBuilder options)
=> options
.LogTo(Console.WriteLine, Microsoft.Extensions.Logging.LogLevel.Information)
.EnableSensitiveDataLogging()
//.UseSqlServer("use a valid connection string here");
.UseSqlite("Data Source=test.db");

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity()
.Property(x => x.MyHex)
.HasConversion(
v => v.ToString("x"),
v => int.Parse(v, System.Globalization.NumberStyles.HexNumber)
);

modelBuilder.Entity()
.Property(x => x.MyDec)
.HasConversion(
v => v.ToString(),
v => int.Parse(v)
);

modelBuilder.Entity().HasData(new MyEntity { Id = 1, MyHex = 16, MyDec = 10 });
modelBuilder.Entity().HasData(new MyEntity { Id = 2, MyHex = 16, MyDec = 16 });
}
}

public class MyEntity
{
public int Id { get; set; }
public int MyHex { get; set; }
public int MyDec { get; set; }

public override string ToString() => $"{Id}({MyHex}, {MyDec})";
}
```

### Stack traces

```text

```

### Verbose output

```text

```

### EF Core version

9.0.1

### Database provider

Microsoft.EntityFrameworkCore.Sqlite

### Target framework

.NET 9.0

### Operating system

Kali Linux

### IDE

Visual Studio Code 1.96.4

EDIT: added one of the relevant issues

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.