Migrating existing payloadbinary from v3 to v7 in AdoNetStorage Postgresql
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 13h 56m
- Merged PRs (30d)
- 351
Description
Upgrading from Orleans **v3.x** to **v7.x** or newer introduces breaking changes in how a grain state being stored in AdoNetStorage, Postgresql in this case.
One of them is that `graintypestring` does not stores an actual grain type anymore, instead it stores hard-coded text either "state" or "PubSubRendezvousGrain" (as mentioned in [this issue](https://github.com/dotnet/orleans/issues/8746)).
I can workaround to produce a new grain state in the new format from the old one.
**Example of grain state in v3**

**Example of grain state in v7+**

The workaround makes the same grain able to read the state from `payloadbinary` but it is not compatible with the way it is being read in v7+.
The error says as below.
> Error from storage provider AdoNetGrainStorage.state during ReadStateAsync for grain consistencysentinel/ConsistencySentinelGrain:00000000-0000-0000-0000-000000000000
>
> Exc level 0: Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: . Path '', line 0, position 0.
> at Newtonsoft.Json.JsonTextReader.ParseValue()
> at Newtonsoft.Json.JsonTextReader.Read()
> at Newtonsoft.Json.JsonReader.ReadAndMoveToContent()
> at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
> at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
> at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
> at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
> at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
> at Orleans.Serialization.OrleansJsonSerializer.Deserialize(Type expectedType, String input) in /_/src/Orleans.Core/Serialization/OrleansJsonSerializer.cs:line 43
> at Orleans.Storage.JsonGrainStorageSerializer.Deserialize[T](BinaryData input) in /_/src/Orleans.Core/Providers/StorageSerializer/JsonGrainStorageSerializer.cs:line 31
> at Orleans.Storage.AdoNetGrainStorage.<>c__DisplayClass31_0`1.b__1(IDataRecord selector, Int32 resultSetCount, CancellationToken token) in /_/src/AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/AdoNetGrainStorage.cs:line 267
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.SelectAsync[TResult](DbDataReader reader, Func`4 selector, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 224
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.ExecuteReaderAsync[TResult](DbCommand command, Func`4 selector, CommandBehavior commandBehavior, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 246
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.ExecuteAsync[TResult](String query, Action`1 parameterProvider, Func`5 executor, Func`4 selector, CommandBehavior commandBehavior, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 284
> at Orleans.Persistence.AdoNet.Storage.RelationalStorage.ReadAsync[TResult](String query, Action`1 parameterProvider, Func`4 selector, CommandBehavior commandBehavior, CancellationToken cancellationToken) in /_/src/AdoNet/Shared/Storage/RelationalStorage.cs:line 165
> at Orleans.Storage.AdoNetGrainStorage.ReadStateAsync[T](String grainType, GrainId grainReference, IGrainState`1 grainState) in /_/src/AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/AdoNetGrainStorage.cs:line 247
> at Orleans.Core.StateStorageBridge`1.ReadStateAsync() in /_/src/Orleans.Runtime/Storage/StateStorageBridge.cs:line 85
So, is it possible to gracefully migrate existing grain states in column `payloadbinary` from **v3** to be compatible with how it is being read in **v7+** to avoid losing all existing states?
Thanks you!
Contributor guide
Research direction
Start with AdoNet/Orleans.Persistence.AdoNet/Storage/Provider/AdoNetGrainStorage.cs at the ReadStateAsync path and inspect Orleans.Core/Providers/StorageSerializer/JsonGrainStorageSerializer.cs plus Orleans.Core/Serialization/OrleansJsonSerializer.cs. Reproduce the v3 payload failure against PostgreSQL, then determine and test a supported migration path that preserves existing grain state when read by v7+.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, postgresql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100