FOLIO-FSE / FOLIO-FSE/folio_migration_tools
Replace In-Memory UUID Dict with SQLite Database
Open
- Dominant language
- Python
- Stars
- 14
- Forks
- 10
- Avg merge
- 55m
- Merged PRs (30d)
- 7
Description
(Discussed at WolfCon 2023):
Currently, the Folio Migration process uses in-memory dictionaries to verify the existence of UUIDs.
Unfortunately, for large data sets, holding this in-memory is causing problems and delays.
We should switch long-term to a flexible database connection option - allowing for distributed task runners to share a database - but first, we can use a local SQLite file to validate.
Contributor guide
Assessment
This issue has not been assessed yet.