dotnet / dotnet/orleans

Migrating existing payloadbinary from v3 to v7 in AdoNetStorage Postgresql

Open
#9,402 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
13h 56m
Merged PRs (30d)
351

Description

Upgrading from Orleans **v3.x** to **v7.x** or newer introduces breaking changes in how a grain state being stored in AdoNetStorage, Postgresql in this case.

One of them is that `graintypestring` does not stores an actual grain type anymore, instead it stores hard-coded text either "state" or "PubSubRendezvousGrain" (as mentioned in [this issue](https://github.com/dotnet/orleans/issues/8746)).

I can workaround to produce a new grain state in the new format from the old one.

**Example of grain state in v3**
![Image](https://github.com/user-attachments/assets/091afe37-73e3-416b-90c0-f3409f711aac)

**Example of grain state in v7+**
![Image](https://github.com/user-attachments/assets/0a671ae4-47b7-4ca5-9068-d1f07beb3037)

The workaround makes the same grain able to read the state from `payloadbinary` but it is not compatible with the way it is being read in v7+.
The error says as below.

> Error from storage provider AdoNetGrainStorage.state during ReadStateAsync for grain consistencysentinel/ConsistencySentinelGrain:00000000-0000-0000-0000-000000000000
>
> Exc level 0: Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: . Path '', line 0, position 0.
> at Newtonsoft.Json.JsonTextReader.ParseValue()
> at Newtonsoft.Json.JsonTextReader.Read()
> at Newtonsoft.Json.JsonReader.ReadAndMoveToContent()
> at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
> at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
> at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
> at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
> at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
> at Orleans.Serialization.OrleansJsonSerializer.Deserialize(Type expectedType, String input) in /_/src/Orleans.Core/Serialization/OrleansJsonSerializer.cs:line 43
> at Orleans.Storage.JsonGrainStorageSerializer.Deserialize[T](BinaryData input) in /_/src/Orleans.Core/Providers/StorageSerializer/JsonGrainStorageSerializer.cs:line 31
> at Orleans.Storage.AdoNetGrainStorage.<>c__DisplayClass31_0`1.b__1(IDataRecord selector, Int32 resultSetCount, CancellationToken token) in /_/src/AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/AdoNetGrainStorage.cs:line 267
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.SelectAsync[TResult](DbDataReader reader, Func`4 selector, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 224
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.ExecuteReaderAsync[TResult](DbCommand command, Func`4 selector, CommandBehavior commandBehavior, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 246
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.ExecuteAsync[TResult](String query, Action`1 parameterProvider, Func`5 executor, Func`4 selector, CommandBehavior commandBehavior, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 284
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.ReadAsync[TResult](String query, Action`1 parameterProvider, Func`4 selector, CommandBehavior commandBehavior, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 165
> at Orleans.Storage.AdoNetGrainStorage.ReadStateAsync[T](String grainType, GrainId grainReference, IGrainState`1 grainState) in /_/src/AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/AdoNetGrainStorage.cs:line 247
> at Orleans.Core.StateStorageBridge`1.ReadStateAsync() in /_/src/Orleans.Runtime/Storage/StateStorageBridge.cs:line 85

So, is it possible to gracefully migrate existing grain states in column `payloadbinary` from **v3** to be compatible with how it is being read in **v7+** to avoid losing all existing states?

Thanks you!

Contributor guide

Open the contributing guide

Research direction

Start with AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/AdoNetGrainStorage.cs at the ReadStateAsync path and inspect Orleans.Core/Providers/StorageSerializer/JsonGrainStorageSerializer.cs plus Orleans.Core/Serialization/OrleansJsonSerializer.cs. Reproduce the v3 payload failure against PostgreSQL, then determine and test a supported migration path that preserves existing grain state when read by v7+.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, postgresql
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.