internetarchive / internetarchive/openlibrary
Provisioning & Deployment Plan
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
This is a planning epic for our [Provisioning and Deployment project](https://github.com/internetarchive/openlibrary/projects/30) and doesn't require direct attention.
-- To be closed when all underlying issues are resolved.
## Top-level Goals
1. [x] #4060 Deprecate & decommission ol-home in favor of ol-home0
1. [x] Migrate legacy ol-home services (cron jobs, ~solr-updater, import-bot~) to run in Docker on ol-home0
2. [x] ~#4382, #4318, #4383, #4395, #4457~ Automate Deployment (on ol-home0)
3. [x] #4060 Final decomission steps for ol-home
5. [x] #4229 Reprovision dev.openlibrary.org like staging.openlibrary.org (py3)
2. [x] #2036 Re-provision all necessary hosts with `focal` (`ol-www1`, `ol-db[1,2]`, `ol-mem[3-5]`, `ol-backup0`)
3. [ ] #4252 Move all necessary hosts to docker (`ol-www1`, maybe `ol-mem[3-5]`)
4. [ ] #4363 Add jenkins to help automate & schedule deploys, branch staging, and solr rebuilding
5. [ ] #4724 Make our deploys create a detailed GitHub release using https://github.com/github-tools/github-release-notes
## Steps
Many of these still require issues & breakdown
- [x] #4324 Move ImportBot to ol-home0 --> make-a-service - Port ID?, Nagios
- [x] Deploy new dockerized `solr-updater` (to prod)
- [x] #4395 Scripts for production deployments -- Automated (4h -> 15min)
- [x] #4457 Reboot Docker services on specified servers
- [x] internetarchive/olsystem#110 Replace references of ol-home with ol-home0
- [x] #4383 Move cron jobs to ol-home0 (Make one service for all cron jobs -- Caution on the double run)
- [x] #4719 rm volume mounts (for deployments) -- needs issue
- [x] #4252 ol-www1 `epic` (edge cases: logs, ssl-termination, sitemaps) @mekarpeles
- [x] #4252 need to spin up a ol-www0 @mekarpeles
- [ ] deploy strategy needs to address ol-www0, solr updater, import bot, cron service + volume_mounts
- [x] #4060 eventual ol-home deprecation
- [ ] remove ol-home proxypass to ol-home0 for infobase + update config
- [ ] #4426 can we dump stats-solr and move from ol-home to ol-home0
- [ ] can we reclaim 4gb of ram from ol-home (until we can decommission)
- [ ] ol-db[1,2] and ol-backup?
### Stakeholders
@bfalling, @cdrini, @cclauss
Contributor guide
Assessment
This issue has not been assessed yet.