dotnet / dotnet/efcore

ComplexCollection + ToJson(): DetectChanges throws "must be initialized to a non-null value" when a nullable nested complex property containing a collection is set to null

Open
#38,632 1 comment 4 reactions 2 assignees Assigned to @AndriySvyryd View on GitHub
area-change-tracking area-complex-types area-json preview-7
Dominant language
C#
Stars
14.8k
Forks
3.4k
PR merge metrics
PR metrics pending

Description

## Bug: ComplexCollection + ToJson() — SaveChangesAsync throws InvalidOperationException "must be initialized to a non-null value" when a nullable nested complex property is set to null

### Description

When an entity has a **`ComplexCollection` mapped with `ToJson()`**, and an item nested inside that JSON graph has a **nullable complex property** whose type itself defines a `List` collection, setting that nullable property to `null` on a tracked entity causes `SaveChangesAsync` to throw:

```
System.InvalidOperationException: The complex type collection 'Root[]Group[]Item.Meta.Entries'
must be initialized to a non-null value before the elements can be accessed.
```

The exception comes from `ChangeTracker.DetectChanges()` itself (before any SQL is generated), in the ordinal-reindex path — `InternalComplexCollectionEntry.RemoveEntry` → `InternalComplexEntry.set_Ordinal` → `GetEntry` walks into the collection of the now-null complex value.

### Trigger conditions (all must hold)

1. Entity has a `ComplexCollection(x => x.Groups, cb => cb.ToJson())` column
2. An item nested in that JSON graph has a **nullable complex property** (`Meta? Meta`)
3. That property's type itself defines a `List` collection (`Entries`)
4. On a tracked entity, the nullable property is set to `null` while its collection holds **2 or more items** — with exactly 1 item the reindex path is not entered and the save succeeds (control scenario included in the attached repro)

### Real-world scenario

This occurs with the standard load → mutate → save pattern: load a tracked entity, remove an optional nested configuration object (set the nullable complex property to `null`) — either by direct assignment or by mapping incoming data onto the tracked instance (Mapperly/AutoMapper/manual) — then call `SaveChangesAsync`. Whenever the removed object's nested collection held 2+ items, the save throws and the update is lost. There is no clean workaround short of intercepting the exception and forcing a full rewrite of the JSON column.

### Steps to reproduce

Full standalone repro project attached: **[20260715_EF_BugRepro.zip](https://github.com/user-attachments/files/30059348/20260715_EF_BugRepro.zip)** (repro + single-entry control + workaround scenarios).

Program.cs — full repro

```csharp
using Microsoft.EntityFrameworkCore;
using Testcontainers.PostgreSql;

var postgres = new PostgreSqlBuilder("postgres:17").Build();
await postgres.StartAsync();
Console.WriteLine($"PostgreSQL started. EF Core version: {typeof(DbContext).Assembly.GetName().Version}");

await using var context = new ReproContext(
new DbContextOptionsBuilder()
.UseNpgsql(postgres.GetConnectionString())
.Options);

var connection = context.Database.GetDbConnection();
await connection.OpenAsync();
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = """
CREATE TABLE "Roots" (
"Id" uuid NOT NULL PRIMARY KEY,
"Groups" jsonb NOT NULL DEFAULT '[]'
)
""";
await cmd.ExecuteNonQueryAsync();
}

// Step 1: Insert entity — the item's Meta has TWO Entries (2+ required to trigger)
var id = Guid.NewGuid();
context.Roots.Add(new Root
{
Id = id,
Groups =
[
new Group
{
Id = Guid.NewGuid(),
Items =
[
new Item
{
Id = Guid.NewGuid(),
Name = "Item 1",
Meta = new Meta
{
Value = 1,
Entries =
[
new Entry { Id = Guid.NewGuid(), RefId = Guid.NewGuid() },
new Entry { Id = Guid.NewGuid(), RefId = Guid.NewGuid() },
]
}
}
]
}
]
});
await context.SaveChangesAsync();
context.ChangeTracker.Clear();

// Step 2: Load tracked entity
var tracked = await context.Roots.SingleAsync(r => r.Id == id);

// Step 3: Set the nullable nested complex property to null
tracked.Groups[0].Items[0].Meta = null;

// Step 4: SaveChangesAsync — CRASHES (with 1 Entry instead of 2 it saves fine)
try
{
await context.SaveChangesAsync();
Console.WriteLine("SUCCESS — no crash.");
}
catch (InvalidOperationException ex) when (ex.Message.Contains("must be initialized to a non-null value"))
{
Console.WriteLine($"BUG REPRODUCED: {ex.Message}");
}
finally
{
await postgres.StopAsync();
}

// ═══ Model ═══

public class Root
{
public Guid Id { get; set; }
public List Groups { get; set; } = [];
}

public record Group
{
public Guid Id { get; set; }
public List Items { get; set; } = [];
}

public record Item
{
public Guid Id { get; set; }
public string Name { get; set; } = "";
public Meta? Meta { get; set; }
}

public record Meta
{
public int Value { get; set; }
public List Entries { get; set; } = [];
}

public record Entry
{
public Guid Id { get; set; }
public Guid RefId { get; set; }
}

public class ReproContext(DbContextOptions options) : DbContext(options)
{
public DbSet Roots { get; set; } = null!;

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity(builder =>
{
builder.HasKey(e => e.Id);
builder.ComplexCollection(e => e.Groups, cb => cb.ToJson());
});
}
}
```

### Expected behavior

`SaveChangesAsync` persists the JSON column with the nulled nested complex property (`"Meta": null`).

### Actual behavior

`SaveChangesAsync` throws:

```
System.InvalidOperationException: The complex type collection 'Root[]Group[]Item.Meta.Entries' must be initialized to a non-null value before the elements can be accessed.
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.InternalComplexCollectionEntry.GetEntry(Int32 ordinal, Boolean original)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.GetComplexCollectionEntry(IComplexProperty property, Int32 ordinal)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalComplexEntry.set_Ordinal(Int32 value)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.InternalComplexCollectionEntry.RemoveEntry(InternalComplexEntry entry, Boolean original)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.InternalComplexCollectionEntry.HandleStateChange(InternalComplexEntry entry, EntityState oldState, EntityState newState)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.OnComplexElementStateChange(InternalComplexEntry entry, EntityState oldState, EntityState newState)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalComplexEntry.OnStateChanged(EntityState oldState)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalComplexEntry.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges, Boolean modifyProperties)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntryBase.SetEntityState(EntityState entityState, Boolean acceptChanges, Boolean modifyProperties, Nullable`1 forceStateWhenUnknownKey, Nullable`1 fallbackState)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.DetectComplexCollectionChanges(InternalEntryBase entry, IComplexProperty complexProperty)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.LocalDetectChanges(InternalEntryBase entry)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.DetectComplexCollectionChanges(InternalEntryBase entry, IComplexProperty complexProperty)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.LocalDetectChanges(InternalEntryBase entry)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.DetectChanges(IStateManager stateManager)
at Microsoft.EntityFrameworkCore.ChangeTracking.ChangeTracker.DetectChanges()
at Microsoft.EntityFrameworkCore.DbContext.TryDetectChanges()
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
```

### Environment

- **EF Core**: tested on 10.0.9 and 10.0.10 — reproduces on both
- **Npgsql.EntityFrameworkCore.PostgreSQL**: 10.0.3
- **Database**: PostgreSQL 17
- **OS**: Windows 11 / .NET 10.0
- **Target framework**: net10.0

### Notes

- With exactly **1 item** in the nested collection the same operation saves fine — the attached repro includes this as a control scenario.
- Also reproduces with `Microsoft.EntityFrameworkCore.Sqlite` — the crash is in change tracking, before any SQL is generated, so it is provider-independent.
- The stored JSON in the database is untouched/valid — the failure is purely client-side change tracking, and the update is lost.
- Workaround (included in the repro, Scenario 3): call `ChangeTracker.DetectChanges()` in a try/catch; on this exception, set `AutoDetectChangesEnabled = false`, mark all `Modified` entries' properties and complex properties as modified (so the full JSON column is rewritten from the current CLR graph), and save with `acceptAllChangesOnSuccess: false` — `AcceptChanges()` also walks the corrupted snapshot and throws.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.