dotnet / dotnet/efcore

"The entity type requires a primary key to be defined" with cross-schema FKs and strongly-typed jsonb mapping

Open
#36,614 2 comments 0 reactions 1 assignee Claimed by @AndriySvyryd View on GitHub
area-model-building customer-reported
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

### Bug description

We have a .NET 9 MVC solution with the following NuGet packages installed:

1. Microsoft.EntityFrameworkCore 9.0.8
2. Microsoft.EntityFrameworkCore.Relational 9.0.8
3. Microsoft.EntityFrameworkCore.Abstractions 9.0.8
4. Microsoft.EntityFrameworkCore.Design 9.0.8
5. Npgsql.EntityFrameworkCore.PostgreSQL 9.0.4

We use PostgreSQL v16 database with 2 schemas - `fit3` and `reports`. There are FKs between the schemas (i.e. a table from `reports` schema has a FK to a table in `fit3` schema).

We use database-first approach, which means that we use `dotnet ef dbcontext scaffold` command to scaffold PostgreSQL tables into C# classes. Some of the columns are of type `jsonb` which is then mapped to strongly-typed C# classes with `OwnsOne`, `OwnsMany`, etc.

Scaffolder is unable to scaffold cross-schema FK relationships. It outputs the warning below. That's why we model the relationship manually with `ForeignKey` and `ReverseProperty` attributes.

```
For foreign key original_test_result_id_fkey on table reports.processed_test_results, unable to model the end of the foreign key on principal table fit3.test_results. This is usually because the principal table was not included in the selection set.
```

We modelled the cross-schema relationship manually like this:

```cs
// table fit.test_result
public partial class TestResult
{
[NotMapped]
public virtual ICollection ProcessedTestResults { get; set; }
}

// table reports.processed_test_result with a FK to fit.test_result
public partial class ProcessedTestResult
{
[ForeignKey(nameof(OriginalTestResultId))]
[InverseProperty(nameof(TestResult.ProcessedTestResults))]
public virtual TestResult OriginalTestResult { get; set; }
}
```

We execute this query:

```cs
var foo = reportsDbContext.ProcessedTestResults.FirstOrDefault();
```

It throws the following exception:

```
System.InvalidOperationException: 'The entity type 'BbCommunication' requires a primary key to be defined. If you intended to use a keyless entity type, call 'HasNoKey' in 'OnModelCreating'. For more information on keyless entity types, see https://go.microsoft.com/fwlink/?linkid=2141943.'
```

If we comment out `ProcessedTestResults` in `TestResult` and `OriginalTestResult` in `ProcessedTestResult`, the `reportsDbContext.ProcessedTestResults.FirstOrDefault()` query finishes successfully.

Why does seemingly unrelated `BbCommunication` cause an exception here? It's some deep strongly-typed class for a JSON object in `fit3.fit_fixtures.custom_config` column. How can we make it work?

### Relevant code parts

`TestResult.partial.cs`:

```cs
public partial class TestResult
{
[NotMapped]
public virtual ICollection ProcessedTestResults { get; set; }

public RuntimeInfo? RuntimeInfoJSON { get; set; }

public TestOutput? TestOutputJSON { get; set; }
}
```

`ProcessedTestResult.partial.cs`:

```cs
public partial class ProcessedTestResult
{
[ForeignKey(nameof(OriginalTestResultId))]
[InverseProperty(nameof(TestResult.ProcessedTestResults))]
public virtual TestResult OriginalTestResult { get; set; }
}
```

`FitDbContext.partial.cs`:

```cs
partial void OnModelCreatingPartial(ModelBuilder modelBuilder)
{
modelBuilder.Entity(entity =>
{
//don't ignore RuntimeInfo, we want to display content exactly as it is stored in the DB - {3CA6957A-8EEF-447E-9A24-6E5CC1ADA123}
//entity.Ignore(x => x.RuntimeInfo);
entity.OwnsOne(x => x.RuntimeInfoJSON, t =>
{
t.ToJson("runtime_info");
t.OwnsOne(y => y.Versions, v =>
{
v.OwnsOne(z => z.Firmware);
v.OwnsOne(z => z.SVN);
v.OwnsOne(z => z.TQSP);
v.OwnsOne(z => z.WinIdea);
});
});

//don't ignore RuntimeInfo, we want to display content exactly as it is stored in the DB - {3CA6957A-8EEF-447E-9A24-6E5CC1ADA123}
//entity.Ignore(x => x.TestOutput);
entity.OwnsOne(x => x.TestOutputJSON, t =>
{
t.ToJson("test_output");
t.OwnsMany(y => y.Measurements, z =>
{
z.OwnsMany(w => w.Data);
});
t.OwnsMany(y => y.MeasurementVars);
});
});

modelBuilder.Entity(entity =>
{
//should ignore CustomConfig, otherwise:
//System.ArgumentException: 'An item with the same key has already been added. Key: [custom_config, Column: fit_fixtures.custom_config (jsonb) NonNullable)]'
//https://github.com/dotnet/efcore/issues/35338
//{3CA6957A-8EEF-447E-9A24-6E5CC1ADA123}
entity.Ignore(x => x.CustomConfig);
entity.OwnsOne(x => x.CustomConfigJSON, cc =>
{
cc.ToJson("custom_config");
cc.OwnsOne(cc_ => cc_.HwSetup, hs =>
{
hs.OwnsMany(hs_ => hs_.DebuggerConnections, dc =>
{
dc.OwnsOne(dc_ => dc_.Debugger);
dc.OwnsMany(dc_ => dc_.DebuggerNodes, dn =>
{
dn.OwnsOne(dn_ => dn_.Board, b =>
{
b.OwnsMany(b_ => b_.SoCs, soc =>
{
soc.OwnsMany(soc_ => soc_.Cores, core =>
{
core.OwnsOne(core_ => core_.CoreBase);
});
});
});
});
});
});
cc.OwnsOne(cc_ => cc_.BbCommunication);
});
});
}
```

`Program.cs`:

```cs
private static void ReproduceBug(IServiceProvider sp)
{
using (var scope = sp.CreateScope())
{
var reportsDbContextFactory = scope.ServiceProvider.GetRequiredService>();
using var reportsDbContext = reportsDbContextFactory.CreateDbContext();
var foo = reportsDbContext.ProcessedTestResults.FirstOrDefault(); // exception is thrown
}
}
```

### Steps to reproduce:

1. Create an empty PostgreSQL v16 database named `EFCoreJson`.
2. Run [db.sql](https://github.com/user-attachments/files/21891427/db.sql) script inside `EFCoreJson` database to create all the tables and relationships between them.
3. Run the [attached solution](https://github.com/user-attachments/files/21891696/EFCoreJson.zip). Place a breakpoint inside `ReproduceBug` method in `Program.cs`. Once `var foo = reportsDbContext.ProcessedTestResults.FirstOrDefault();` is executed, an exception is thrown. No `SELECT` SQL query is logged to the console which indicates that the exception occurs even before the query is acutally executed, perhaps in the stage where EF Core sets up its behind-the-scenes architecture.
4. If you remove `TestResult.ProcessedTestResults` and `ProcessedTestResult.OriginalTestResult` properties (manually configured cross-schema relationship), the query works normally.

### Stack traces

```text
An unhandled exception of type 'System.InvalidOperationException' occurred in Microsoft.EntityFrameworkCore.dll: 'The entity type 'BbCommunication' requires a primary key to be defined. If you intended to use a keyless entity type, call 'HasNoKey' in 'OnModelCreating'. For more information on keyless entity types, see https://go.microsoft.com/fwlink/?linkid=2141943.'
at Microsoft.EntityFrameworkCore.Infrastructure.ModelValidator.ValidateNonNullPrimaryKeys(IModel model, IDiagnosticsLogger`1 logger)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelValidator.Validate(IModel model, IDiagnosticsLogger`1 logger)
at Microsoft.EntityFrameworkCore.Infrastructure.RelationalModelValidator.Validate(IModel model, IDiagnosticsLogger`1 logger)
at Npgsql.EntityFrameworkCore.PostgreSQL.Infrastructure.Internal.NpgsqlModelValidator.Validate(IModel model, IDiagnosticsLogger`1 logger)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelRuntimeInitializer.Initialize(IModel model, Boolean designTime, IDiagnosticsLogger`1 validationLogger)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.CreateModel(DbContext context, ModelCreationDependencies modelCreationDependencies, Boolean designTime)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.GetModel(DbContext context, ModelCreationDependencies modelCreationDependencies, Boolean designTime)
at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel(Boolean designTime)
at Microsoft.EntityFrameworkCore.Internal.DbContextServices.get_Model()
at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.b__8_4(IServiceProvider p)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies()
at Microsoft.EntityFrameworkCore.DbContext.get_ContextServices()
at Microsoft.EntityFrameworkCore.DbContext.get_Model()
at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.get_EntityType()
at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.CheckState()
at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.get_EntityQueryable()
at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.System.Linq.IQueryable.get_Provider()
at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source)
at EFCoreJson.Web.Program.ReproduceBug(IServiceProvider sp) in C:\Play\EFCoreJson\EFCoreJson.Web\Program.cs:line 50
at EFCoreJson.Web.Program.Main(String[] args) in C:\Play\EFCoreJson\EFCoreJson.Web\Program.cs:line 20
```

### EF Core version

9.0.8

### Database provider

Npgsql.EntityFrameworkCore.PostgreSQL

### Target framework

.NET 9.0

### Operating system

Windows 11

### IDE

Visual Studio Professional 2022 17.14.9

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.