conda-forge / conda-forge/conda-forge.github.io
Infrastructure update for Channel Cloning/CDN sync
- Dominant language
- JavaScript
- Stars
- 170
- Forks
- 320
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 5
Description
### Your question:
Hey everyone,
we (Anaconda) have been working on upgrades for the channel cloning/CDN sync infrastructure of anaconda.org, which is is used for conda-forge.
## Time planning
We are planning to switch to the new infrastructure tomorrow, 2024-08-27 at 12:30 UTC.
**No downtime is expected for users**.
Due to the amount of changes on the conda-forge channel, the initial sync after the switch to the new infrastructure will be a bit slower than the usual syncs - it is currently running in a "dry run" mode on the new infrastructure already to keep its caches up to date, and I expect the initial sync on the new infrastructure to take ~30 minutes.
We have already implemented this change for the r and anaconda channels to ensure that our maintenance process and the new infrastructure are working as desired.
## What's changing?
* The codebase has been updated to make the cloning more fault-tolerant and reliable
* The new infrastructure is more tightly integrated into our observability stack, enabling us to detect problems earlier and more granularly (e.g. a specific package that fails to clone instead of just a general cloning error)
* We're adding a `last-updated.json` file at the channel root, which contains the last update time both as UNIX and ISO8601 timestamps. We are using this directly to monitor the updates end-to-end. Check it out for the [anaconda](https://conda.anaconda.org/anaconda/last-updated.json) channel if you're interested!
## Other notes
This topic was discussed in the conda-forge meeting on 2024-08-21 and it was decided to bring it up here to raise awareness.
If there's any questions, please ask!
Contributor guide
Research direction
The issue is an infrastructure announcement rather than a contribution task and names no files, tests, or entry points. Start by confirming whether the planned switch has already happened and locating the channel cloning/CDN sync and observability configuration. No newcomer-sized definition of done is provided.
Written by the indexing model from the issue text.
Assessment
- Domain
- infrastructure, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 10/100