dotnet / dotnet/efcore

Unexpected behaviour on TemporalAll GroupBy

Open
#32,048 1 comment 0 reactions 0 assignees View on GitHub
area-query customer-reported
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

I am currently using EF Core with SqlServer temporal tables and I noticed an unexpected behaviour when using TemporalAll and then GroupBy.

When you group by and select all the groups, on each collection value of the IGrouping if you try a Count() you'll get the correct number of all the versions in the table for each group, but once projected they only contain one version (the current state).

At the same time by materializing it with a ToList, all the rows are returned, so by proceeding with the GroupBy in memory works as expected.

Example:
This doesn't return all the rows even if the Count is correct
```cs
var set = await
context
.Set()
.TemporalAll()
.Select(e => new
{
Employee = e,
ValidFrom = EF.Property(e, "ValidFrom"),
ValidTo = EF.Property(e, "ValidTo")
})
.GroupBy(r => r.Employee.EmployeeId)
.Select(t => new
{
Id = t.Key,
Count = t.Count(),
Versions = t.OrderByDescending(v => v.ValidTo).Select(v => new
{
v.Employee.EmployeeId,
v.Employee.Name,
v.Employee.Position,
v.Employee.Department,
v.Employee.Address,
v.Employee.AnnualSalary,
v.ValidFrom,
v.ValidTo,
}).ToList()
})
.ToListAsync(cancellationToken);
```

This by materializing with a ToList will return all the rows
```cs
var set = await
context
.Set()
.TemporalAll()
.Select(e => new
{
Employee = e,
ValidFrom = EF.Property(e, "ValidFrom"),
ValidTo = EF.Property(e, "ValidTo")
})
.ToListAsync(cancellationToken);

var result = set
.GroupBy(r => r.Employee.EmployeeId)
.Select(t => new
{
Id = t.Key,
Count = t.Count(),
Versions = t.OrderByDescending(v => v.ValidTo).Select(v => new
{
v.Employee.EmployeeId,
v.Employee.Name,
v.Employee.Position,
v.Employee.Department,
v.Employee.Address,
v.Employee.AnnualSalary,
v.ValidFrom,
v.ValidTo,
}).ToList()
})
.ToList();
```

Is the GroupBy only collecting the current state table instead of using all the versions?
Is this a correct behaviour?

Consider the following [gist](https://gist.github.com/cfgiugliano/4016827cfc0ec5346b3b195209e81b9c) as a working example

Use the DoubleSalaries api a couple of times to generate some historical data, then use the GetEmployeesVersions_DBGroupBy api to get the data.

This is the generated query

```sql
SELECT [t].[EmployeeId], [t].[c], [e0].[EmployeeId], [e0].[Name], [e0].[Position], [e0].[Department], [e0].[Address], [e0].[AnnualSalary]
FROM (
SELECT [e].[EmployeeId], COUNT(*) AS [c]
FROM [Employees] FOR SYSTEM_TIME ALL AS [e]
GROUP BY [e].[EmployeeId]
) AS [t]
LEFT JOIN [Employees] FOR SYSTEM_TIME ALL AS [e0] ON [t].[EmployeeId] = [e0].[EmployeeId]
ORDER BY [t].[EmployeeId], [e0].[ValidTo] DESC
```

which once run on DB will return all the rows
```
A9A26C83-CB03-41BB-879D-59541DBC1BDD 4 A9A26C83-CB03-41BB-879D-59541DBC1BDD John Engineer .NET Somewhere 24000.00
A9A26C83-CB03-41BB-879D-59541DBC1BDD 4 A9A26C83-CB03-41BB-879D-59541DBC1BDD John Engineer .NET Somewhere 12000.00
A9A26C83-CB03-41BB-879D-59541DBC1BDD 4 A9A26C83-CB03-41BB-879D-59541DBC1BDD John Engineer .NET Somewhere 6000.00
A9A26C83-CB03-41BB-879D-59541DBC1BDD 4 A9A26C83-CB03-41BB-879D-59541DBC1BDD John Engineer .NET Somewhere 3000.00
72F6137D-1CB2-488F-B49D-98C4493332E6 4 72F6137D-1CB2-488F-B49D-98C4493332E6 Steve Engineer .NET Somewhere 8000.00
72F6137D-1CB2-488F-B49D-98C4493332E6 4 72F6137D-1CB2-488F-B49D-98C4493332E6 Steve Engineer .NET Somewhere 4000.00
72F6137D-1CB2-488F-B49D-98C4493332E6 4 72F6137D-1CB2-488F-B49D-98C4493332E6 Steve Engineer .NET Somewhere 2000.00
72F6137D-1CB2-488F-B49D-98C4493332E6 4 72F6137D-1CB2-488F-B49D-98C4493332E6 Steve Engineer .NET Somewhere 1000.00
```

### Include provider and version information

EF Core version: 7.0.12
Database provider: Microsoft.EntityFrameworkCore.SqlServer 7.0.12
Target framework: .NET 6.0
Operating system: Windows 10 22H2
IDE: Visual Studio 2022 17.8.0 Preview 3

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.