dotnet / dotnet/orleans

Deserializing record types created with Orleans 7.0 is not working on Orleans 7.1

Open
#8,310 0 comments 0 reactions 1 assignee Claimed by @ReubenBond View on GitHub
area-serialization Needs: triage :mag:
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
14h 42m
Merged PRs (30d)
354

Description

We shipped an update to a project using Orleans 7.0 yesterday, and noticed two issues after a while:

- Some grains had their state reset, they essentially had the equivalent of a `new()`-ed up state as their internal state. I can find no apparent errors/warnings in logs about this.
- Some grains are failing to activate due to state deserialization errors.

The exception we see when trying to deserialize the state is:

```
Unhandled exception. Orleans.Serialization.UnsupportedWireTypeException: A WireType value of TagDelimited is expected by this codec. [LengthPrefixed, IdDelta:1, SchemaType:Expected]
at Orleans.Serialization.WireProtocol.Field.UnsupportedWireType() in /_/src/Orleans.Serialization/WireProtocol/Field.cs:line 234
at Orleans.Serialization.Codecs.DateTimeOffsetCodec.ReadValue[TInput](Reader`1& reader, Field field) in /_/src/Orleans.Serialization/Codecs/DateTimeOffsetCodec.cs:line 44
at Orleans.Serialization.Serializer.Deserialize[T](ReadOnlySpan`1 source) in /_/src/Orleans.Serialization/Serializer.cs:line 389
at Orleans.Serialization.Serializer.Deserialize[T](ReadOnlyMemory`1 source) in /_/src/Orleans.Serialization/Serializer.cs:line 432
at Orleans.Storage.OrleansGrainStorageSerializer.Deserialize[T](BinaryData input) in /_/src/Orleans.Core/Providers/StorageSerializer/OrleansGrainStateSerializer.cs:line 35
at Orleans.Storage.GrainStorageSerializerExtensions.Deserialize[T](IGrainStorageSerializer serializer, ReadOnlyMemory`1 input) in /_/src/Orleans.Core/Providers/IGrainStorageSerializer.cs:line 43
```

Our record types essentially look like this:

```cs
[GenerateSerializer]
[Immutable]
public sealed record MyType
{
[Id(0)] public string Value { get; init; }
}
```

After a [discussion on discord](https://discord.com/channels/333727978460676096/1073243186165715034), what we noticed was that if we change the serializer attribute to `[GenerateSerializer(IncludePrimaryConstructorParameters = false)]` when running on Orleans 7.1, we were again able to deserialize our state payloads that were written using Orleans 7.0. In this scenario, however, we were no longer able to deserialize state payloads that were written using Orleans 7.1.

I have created a quick reproduction https://github.com/tanordheim/Orleans71SerializerWoes. This code doesn't throw an exception when deserializing, but it does silently fail to deserialize the payload and return an empty object which is part of the problem we saw. This can be tested using that sample repository by:

- Run: `dotnet run --project SerializePayload/SerializePayload.csproj`. This will generate a `payload.bin` of the test record containing the serialized output using Orleans 7.0.
- Run: `dotnet run --project DeserializePayload/DeserializePayload.csproj`. This will deserialize `payload.bin` into the test record and print the value using Orleans 7.0. This works.
- Change the Orleans SDK version requirement in `Types/Types.csproj` to `7.1.0`.
- Re-run: `dotnet run --project DeserializePayload/DeserializePayload.csproj`. This will print an empty value, seemingly having failed to deserialize the payload silently.

I'll look at creating a reproduction that also throws an exception during deserialize tomorrow, hopefully this is enough to get some ideas rolling on what could be wrong. We have a production incident right now since we rolled this out with insufficient testing, incorrectly assuming that serialization between minor versions of Orleans would be problem free. We'll start evaluating if we need to do some kind of migration on our production data to get it up to a proper 7.1 format tomorrow.

Potential related PRs are https://github.com/dotnet/orleans/pull/8289 according to the conversations we had on Discord.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.