Tool to migrate data between SpatialData versions
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 394
- Forks
- 95
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 7
Description
Is your feature request related to a problem? Please describe.
When the specification of SpatialData is changed, existing datasets do not have these changes, and under circumstances may even become incompatible.
For the in-memory representation, the library's reader functions support reading older versions (in most cases). However, users may want to upgrade the on-disk data to the latest version. Another case is when something is not covered by backward-compatibility of readers, e.g. due to errors in the data (#655).
Describe the solution you'd like
As discussed earlier, we want a tool for migrating data, if more format changes occur in future. I open this new issue for tracking this, since #655 was too specific and is closed.
Describe alternatives you've considered
- Relying on backward-compatible readers, then using
writeto save in the latest format: Some cases may not be covered by readers. - Adding a new "upgrade" function to the
spatialdatalibrary: Migration is not a frequent use case and would bloat the library, especially if supporting special cases (one-time issues, erronous data). Additionally, backward-compatibility of certain features can be easily deprecated in the library, while still preserving it in a separate tool. - Separate
spatialdata-migratetool
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the migration requirements in this issue and the related case in #655. Compare the proposed separate spatialdata-migrate tool with relying on backward-compatible readers and write; done would mean a defined migration approach for upgrading on-disk SpatialData across specification versions, including unsupported or erroneous data cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100