AdoNetGrainStorage HashPicker customization support
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 13h 56m
- Merged PRs (30d)
- 351
Description
Hi,
I have several clusters powered by Orleans v3.x with ADO.NET grain persistence (MySQL/PostgreSQL, payload in json/jsonb column) and want to upgrade them to v8.2+ with reasonable downtime (shutdown cluster -> deploy new version) and without persistent state loss. To achieve this goal I've tried to:
* switch to new packages, annotate DTO and handle another changes described in [migration guide](https://learn.microsoft.com/en-us/dotnet/orleans/migration-guide)
* set correct state names in `PersistentStateAttribute`
* slightly change Read/Write/Clear queries as I need to keep storing grain state payload in json column (for better readability and indexing support)
And it's almost here but with one problem: `GrainIdHash` and `GrainTypeHash` mismatch.
As I can see hashing algorithm was changed from JenkinsHash to xxHash32 in v7 (https://github.com/dotnet/orleans/pull/7949). So, to complete the upgrade we need to handle this hashing change. Changing values in database seems to be too hard (big table without primary key, unable to compute new hashes just using SQL). But we have another way to do it: customizing `HashPicker` in `AdoNetGrainStorage`. Unfortunately, this property can't be simply configured via dependency injection and the only way to do it is something like that:
```c#
siloBuilder.AddAdoNetGrainStorageAsDefault();
var storageDescriptor = siloBuilder.Services.LastOrDefault(static d
=> d.IsKeyedService && d.ServiceType == typeof(IGrainStorage));
if (storageDescriptor is null)
throw new InvalidOperationException("Unable to find IGrainStorage service descriptor.");
if (storageDescriptor.KeyedImplementationFactory is null)
throw new InvalidOperationException("Unexpected IGrainStorage service descriptor content.");
siloBuilder.Services.Remove(storageDescriptor);
siloBuilder.Services.Add(ServiceDescriptor.KeyedSingleton(
storageDescriptor.ServiceType,
storageDescriptor.ServiceKey,
(provider, key) =>
{
var storage = storageDescriptor.KeyedImplementationFactory.Invoke(provider, key);
if (storage is not AdoNetGrainStorage adoNetGrainStorage)
throw new InvalidOperationException("Unexpected IGrainStorage service implementation type.");
adoNetGrainStorage.HashPicker = new StorageHasherPicker([new Orleans3Hasher()]);
return adoNetGrainStorage;
}));
```
Although it works (upgraded cluster behaves as expected, existing grain state is available), this workaround is not reliable as it depends on internal Orleans implementation and could stop working after future updates.
Taking into account the above, I propose the following changes:
```c#
public class AdoNetGrainStorageOptions
{
//
public IStorageHasherPicker HashPicker { get; set; }
}
public class AdoNetGrainStorage
{
public AdoNetGrainStorage(
// other parameters
IOptions options)
{
//other assignments
this.HashPicker = options.Value.HashPicker;
}
}
```
This new property should have the same behavior as `AdoNetGrainStorageOptions.GrainStorageSerializer` with setting default value via `PostConfigure` if it's `null`.
Also it would be nice to add public Orleans v3-compatible `IHasher` implementation to keep customers from copy-paste JenkinsHash-based hasher.
In addition, I suppose this v3->v7 hashing change should be described in migration guide as its handling is a required step to migrate existing cluster without data loss.
Contributor guide
Research direction
Review AdoNetGrainStorageOptions and the AdoNetGrainStorage constructor, then trace the existing GrainStorageSerializer configuration and its PostConfigure setup. Verify that HashPicker can be configured without replacing the service registration, assess the public Orleans v3-compatible hasher, and update the migration guide to cover the hashing change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, mysql, postgresql
- Domain
- databases, distributed-systems, documentation
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100