internetarchive / internetarchive/openlibrary
Followup to Psql 18 Migration
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Feature Request
## Problem / Opportunity
https://github.com/internetarchive/openlibrary/issues/5675#issuecomment-4136966418
## Next Steps
1. [x] Add `olsystem` to ol-db0 and consider deploy flow changes for `ol-db0` (up-to-date olsystem)
2. [x] P0 We need to now setup ol-db3 as replica
3. We still have indices we may want to change (e.g. int -> bigint) #10921 #12009
4. We need to now setup ol-db3 as replica
5. We need to update our cron jobs to move backups to *not* ol-backup0??? (which means we need an ol-backup1?)
6. Some plan for decomissioning ol-db1 + ol-db2; cc @abezella
7. ol-db0 **extremely carefully** `sudo rm -rf /2/postgresql/9.3/main` and uninstall 9.3: `sudo apt-get remove postgresql-9.3 postgresql-client-9.3`
8. symlink pg files from olsystem (and ferm as well). NB: the pg files in /2/ contain different path references than /etc and so symlink may not be possible. Either way, ground truth files should be version controlled in olsystem, maybe for both.
9. Update grafana + graphite stats e.g. https://catalogd.archive.org/control/stats/db.php?db=openlibrary#60d
## Proposal
### Breakdown
Implementation Details (for maintainers)
#### Related files
Refer to [this map of common Endpoints](https://docs.openlibrary.org/developers/backend/endpoints.html):
*
#### Requirements Checklist
Checklist of requirements that need to be satisfied in order for this issue to be closed:
* [ ]
#### Stakeholders
*
#### Instructions for Contributors
* **Before** [creating a new branch](https://docs.openlibrary.org/developers/tools/git.html#making-changes-and-creating-a-pull-request) or pushing up changes to a PR, please first [run these commands](https://docs.openlibrary.org/developers/tools/git.html#working-on-your-branch) to ensure your repository is up to date, as the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Research direction
Start by reviewing the linked Psql 18 migration discussion and the olsystem configuration for ol-db0 and ol-db3. Then inventory the remaining checklist items covering replication, cron backups, database decommissioning, PostgreSQL file paths, and Grafana/Graphite statistics; completion requires those operational changes to be checked off.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grafana, postgresql
- Domain
- databases, infrastructure, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100