An alternative way to migrate Grain state storage from Orleans 3.x to 7
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 14h 42m
- Merged PRs (30d)
- 354
Description
I was reading the [PR](https://github.com/dotnet/orleans/pull/8214) for migrating the state from Orleans 3.x to 7.
From what I understood and discussed with @ReubenBond and @benjaminpetit , the process boils down to having the `GrainReferenceExtractor` to ultimately pull info from the `GrainReference` in order to use the `GrainIdKeyExtensions` methods to create the `IdSpan` (the new Grain id).
The whole process works by having the grains to activate, load the current state, patch the Id to the new format and save it back. Assuming the state is using JSON, no changes are needed to the state payload itself.
The problem is that it requires all grains with state to be activated somehow. In a system with massive number of grains it is unreasonable to expect all grains to be activated at once.
I was looking for a way to perform the migration as one step fully offline migration.
The pseudo-steps would be as following:
1. Scan the whole storage for the PK and RK. They are essentially the old grain id format.
2. Build the new `IdSpan` using that old grain id.
3. Build a Map of old grain id to new grain id.
With that map in hand, we are free to use whatever tool to migrate the data. The migration tool would read the old grain id, look it up in the map and write the new grain id along with the state payload on the new storage account. For my particular case, I would use Azure Data Factory to do the copy based on the map.
To achieve this, we need to be able to "Parse" the old grain id. So, can you guys point out how can we do that?
Once we have this info I can put a PR together with an alternative way to generate the migration map as a CLI tool or something. This way people can build this list and use whatever process they want to actually copy the data using that map.
I understand the reasoning behind building the "in-place" upgrade of the storage as it was on that PR. But I think what I'm proposing should be a simpler yet more flexible way to do this migration. Orleans would basically provide a tool to build the map. The copy/migration work would be done in whatever way fits best the end user use case freeing Orleans itself on having to support multiple migration paths out there.
Contributor guide
Assessment
This issue has not been assessed yet.