Automated testing of deployment in pipeline.
- Dominant language
- Python
- Stars
- 65
- Forks
- 11
- Avg merge
- 21h 25m
- Merged PRs (30d)
- 70
Description
**Describe the task**
We need automated testing of our deployed products, as part of our github actions, as a blocker to deploying.
E.g. Automated test that checks https://wps-pr-xxxx.apps.silver.devops.gov.bc.ca/hfi-calculator, ensure that it's working correctly, until such time as it return 200, don't allow deploy to production.
After deploying to production, check https://psu.nrs.gov.bc.ca/hfi-calculator, until such time as it returns 200, don't allow merging code.
**Acceptance Criteria**
`After dev deployment completes`:
- [ ] Percentile Calculator - cannot deploy to prod if test in dev does not pass.
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/percentile-calculator` returns 200
- [ ] MoreCast
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/morecast` returns 200
- [ ] C-Haines
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/c-haines` returns 200
- [ ] FireBat
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/fire-behaviour-calculator` returns 200
- [ ] FireBat bookmark
- [ ] GET `https://wps-pr-1792.apps.silver.devops.gov.bc.ca/fire-behaviour-calculator?s=266&f=c5&c=NaN&w=20,s=286&f=c7&c=NaN&w=16,s=1055&f=c7&c=NaN&w=NaN,s=305&f=c7&c=NaN&w=NaN,s=344&f=c5&c=NaN&w=NaN,s=346&f=c7&c=NaN&w=NaN,s=328&f=c7&c=NaN&w=NaN,s=1399&f=c7&c=NaN&w=NaN,s=334&f=c7&c=NaN&w=NaN,s=1082&f=c3&c=NaN&w=NaN,s=388&f=c7&c=NaN&w=NaN,s=309&f=c7&c=NaN&w=16,s=306&f=c7&c=NaN&w=NaN,s=1029&f=c7&c=NaN&w=NaN,s=298&f=c7&c=NaN&w=NaN,s=1108&f=c5&c=NaN&w=NaN,s=836&f=c7&c=NaN&w=NaN#state=2ec784ca-c46a-49d0-b2b3-1cf32a9015a2&session_state=7d9447c8-db66-4661-b4cb-03d2ac0d1d8f&code=32292df4-2bdf-4f90-a4a8-c8dbcda682a9.7d9447c8-db66-4661-b4cb-03d2ac0d1d8f.2b63f390-f3dc-43ae-89f2-016453863476` returns 200
- [ ] Fire Behaviour Advisory
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/fire-behaviour-advisory` returns 200
- [ ] HFI Calculator
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/hfi-calculator` returns 200
- [ ] FWI Calculator
- [ ] GET `https://wps-pr-.apps.silver.devops.gov.bc.ca/fwi-calculator` returns 200
`After prod deployment completes`:
- [ ] All of the above checks, replacing the PR URL with https://psu.nrs.gov.bc.ca/
**Additional context**
- Also known as a smoke test. See this post for a step-by-step coded solution: https://circleci.com/blog/smoke-tests-in-cicd-pipelines/
- Tests must not alter operational data in production. E.g. a test that checks if the production deployment work, can't go doing updates in production, it must be limited to read operations in as far as is possible.
- Tests may alter data in dev/test environments - but i.m.h.o. we should be running the same set of tests in production, dev and test.
- Our first tests should be dead simple - look for 200 OK - and leave it at that.
- Whatever solution we pick, it should be easy to build on it to introduce more complicated integration tests. Our first pass doesn't have to have some fancy integration testing toolset, it just has to be be able to grow into that.
- One ALWAYS need to do some manual testing when deploying, this testing does not remove that step.
- We have had very few instances of accidental deployment of broken instances to production, the cost of that downtime should be measured against the cost of implementing this feature.
Contributor guide
Research direction
Start by reading the repository's GitHub Actions deployment pipeline and identify the dev and production deployment stages. Run the listed application URLs and verify the expected HTTP 200 responses, including the FireBat bookmark URL. Done means all listed smoke checks gate the next deployment or merge without changing production data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100