FOLIO-FSE / FOLIO-FSE/folio_migration_tools

Replace In-Memory UUID Dict with SQLite Database

Open
#670 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14
Forks
10
Avg merge
55m
Merged PRs (30d)
7

Description

(Discussed at WolfCon 2023):

Currently, the Folio Migration process uses in-memory dictionaries to verify the existence of UUIDs.

Unfortunately, for large data sets, holding this in-memory is causing problems and delays.

We should switch long-term to a flexible database connection option - allowing for distributed task runners to share a database - but first, we can use a local SQLite file to validate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.