dotnet / dotnet/runtime

XmlSerializer: reflection-based reader returns null for a nil collection root where the IL-generating reader returns empty

Open
#132,762 0 comments 0 reactions 0 assignees View on GitHub
area-Serialization
Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Description

(Found when investigating PR #132356 for issue #66264 ...)

### Description

The reflection-based `XmlSerializer` reader and the IL-generating reader disagree on what a nil collection **root** deserializes to. The IL-generating reader materializes an empty collection; the reflection-based reader returns `null`.

At member level the two agree (both produce an empty collection). Only the root differs.

This matters beyond test configurations: `XmlSerializer.Mode` resolves to `ReflectionOnly` whenever `RuntimeFeature.IsDynamicCodeSupported` is false, so the reflection reader is the only one NativeAOT ever uses.

### Reproduction Steps

```csharp
using System;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
using System.Xml.Serialization;

// Simulate IsDynamicCodeSupported == false, which is what NativeAOT gets.
// Same technique as src/libraries/System.Private.Xml/tests/TrimmingTests/XmlSerializer.Deserialize.SealerOpt.cs
const int ReflectionOnly = 1;
typeof(XmlSerializer).GetField("s_mode", BindingFlags.NonPublic | BindingFlags.Static)
.SetValue(null, ReflectionOnly);

string xml = """


""";

var serializer = new XmlSerializer(typeof(List));
object result = serializer.Deserialize(new StringReader(xml));

Console.WriteLine(result is null ? "null" : $"empty, Count={((List)result).Count}");
```

Delete the `s_mode` line to exercise the IL-generating reader instead.

### Expected behavior

Both readers produce the same result. The IL-generating reader's behavior is the long-standing one, so `empty, Count=0`.

(Whether "nil should mean empty" is the right rule at all is a separate and arguable question. It is what `XmlSerializer` has always done for collections it populates in place, so the two backends agreeing on it is the goal here, not changing it.)

### Actual behavior

| Reader | Result |
| --- | --- |
| IL generating | `empty, Count=0` |
| Reflection based | `null` |

### Regression?

No. Pre-existing, and present as far back as the reflection-based reader.

### Known Workarounds

Null-check the result at the call site, or avoid `xsi:nil` on a collection root.

### Configuration

Not configuration specific, but only observable where the reflection-based reader runs: NativeAOT and any other configuration with dynamic code disabled, or when `XmlSerializer.Mode` is set to `ReflectionOnly` directly.

### Other information

- Measured while working on #132356. The test `Xml_NilCollectionAsRoot_BecomesEmpty` in `src/libraries/System.Private.Xml/tests/XmlSerializer/XmlSerializerTests.cs` excludes `List` behind `#if !ReflectionOnly` for exactly this reason, with a comment pointing at it.
- The IL-generating reader creates an empty collection for a nil element as a deliberate, long-standing behavior. The reflection reader reproduces that for members but not for a root, which suggests the end-of-collection "create it anyway" step is not reached on the root path. This may be the same underlying gap as the reflection reader's handling of `XmlMembersMapping`. Not verified.
- Worth knowing for whoever picks this up: on #132356's branch, read-only collections created through `[CollectionBuilder]` **do** produce empty at a nil root under both readers, because the hydration step always runs. So the divergence is specific to collections that are populated in place.

> [!NOTE]
> This issue was drafted with GitHub Copilot.

Contributor guide

Open the contributing guide

Research direction

Start with src/libraries/System.Private.Xml/tests/XmlSerializer/XmlSerializerTests.cs and the Xml_NilCollectionAsRoot_BecomesEmpty test, then run it in ReflectionOnly mode using the issue's reproduction. Trace the reflection-based reader's root collection path and compare it with the member-level behavior. Done means nil collection roots produce an empty collection consistently in both readers, with coverage for the reflection path.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.