Revisit monitoring / status page
- Dominant language
- Python
- Stars
- 15
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
Following #701, we've realized we can probably simplify status.kiwix.org content and enhance monitoring.
# Problem statement
We currently have in UptimeRobot a mix of fine-grained monitors (like for download or library where we monitor each components with a dedicated monitor) and high-level ones (like for CMS, Imager-Service, Zimfarm where we monitor a "health-check" URI which in turn monitor individual system components).
Downsides are two-fold:
- on one side, we do not have advanced UptimeRobot monitoring capabilities (e.g. SSL certificate expiration) on individual components (e.g. CMS API)
- on the other side, we do not have an "aggregated" monitor to expose on status.kiwix.org and we end-up with details which are not relevant to users of this page (we can choose which monitor we expose on status page but there is not single good choice)
# Proposition
We (@rgaudin and @benoit74) propose enhance the situation by performing two moves.
First, we should add all publicly exposed HTTP endpoints to Uptime Robot to benefit from its monitoring capabilities and get better monitoring "depth". These monitor should not be exposed on status.kiwix.org
Second, we should deploy a healthcheck for download service, and another one for the library service. This healthcheck will (like other healthcheck) monitor in turn the various endpoints / details of the service. These healthcheck can then be added to the status page. These healthcheck will run like CMS and Zimfarm healthcheck on `bastion` node, so they are mostly independent of service monitored. I wonder if it is the right moment to create a shared healthcheck container image. I will have to dive again in to current CMS healthcheck to see if we can abstract its logic.
@kelson42 WDYT?
Contributor guide
Research direction
Start by reviewing the current CMS healthcheck running on the bastion node, along with the existing UptimeRobot monitors and status.kiwix.org configuration. Done means publicly exposed endpoints have suitable monitors, download and library have aggregated healthchecks, and the appropriate healthchecks are exposed on the status page.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- devops, infrastructure, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100