Cosmos: Handle late discriminators in materialization
- Dominant language
- C#
- Stars
- 14.8k
- Forks
- 3.4k
- PR merge metrics
- PR metrics pending
Description
### What problem are you trying to solve?
Since #38550 cosmos materialization is done in a streaming fashion, using Utf8JsonReader.
This is faster and allocation light. Especially when comparing the to previous materializer that used Newtonsoft's DOM as an intermediate step.
However, when a structural type has an inheritance hierarchy, the materializer needs to know the discriminator value before it can start reading other values. Similar to STJ, the materializer will do a scan over the document to find the discriminator property and it's value. A document could be as large as 2MB and the discriminator can appear anywhere in the document. [Limited benchmarking](https://github.com/JoasE/ModernizeDeserializationBenchmark) has shown that scanning the document can be slow when the discriminator appears late, to the point where STJ's DOM would be faster with very little increase in GC pressure.
The serializer has been changed to put discriminators early in the document, but using different write method or migrating from an older version of EF, we don't know the location of the discriminator
### Describe the solution you'd like
To discuss.
STJ actually throws an **opt-out** exception when the discriminator is not the first property in a json document that has an inheritance hierarchy. We could consider the same, or even switch to DOM when we can't find the discriminator in the first (few) properties. This would mean maintaining 2 methods of materialization tho, which probably isn't worth it. Completely switching to STJ DOM could be an option, but we would be slightly slowing down the happy path materialization.
Contributor guide
Research direction
Start by reviewing the streaming Cosmos materialization introduced by #38550, focusing on the Utf8JsonReader discriminator scan and the serializer's discriminator ordering. Compare late-discriminator behavior with the mentioned STJ DOM and opt-out exception approaches; done requires a decided strategy and corresponding performance and correctness validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100