Managing one-time persistent data fixes

Hopefully it is rare, but one-time fixes may be required to correct data in what ever is used for persistent storage, e.g. a relational database like postgresql. Data can change as the business and customers change, or our understanding of their needs. Issues can arise where incorrect

Migration on service startup

Focus on schema design changes only when running migration scripts during the startup process of a Clojure application.

Avoid data loading and data fixes, which tend to be one-off issues

focused on changes only to the database design and not the values within the persistent store.

Data loading & data fixes

The application may already have an interface for customers or administrators to fix issues, however, if there is no such interface what are the effective ways to apply changes to the values in the persistent store and how can they be effectively managed?

Options

• sql queries run directly - need to document a repeatable manually run process for each environment
• manual migrations script - separate scripts from those that always run at startup and are run manually by engineering team or operations team to fix a specific data related issue.
• Writing an Administrative or Customer facing UI - a lot of investment if this is just for occasional data fixes
• Using Clojure CLI -X to add data fix functions to a project - part of the project so the code is effectively manage and can be automated as a CI task and called when ever required. Also simple to migrate between environments as its the same kind of process as other development workflows.

The issue names no files or tests. Start by reviewing the “Migration on service startup” and “Data loading & data fixes” sections, then compare the listed SQL, manual migration, UI, and Clojure CLI -X options. Done should be a documented, repeatable approach for applying one-time persistent-data fixes across environments.