openshift / openshift/origin

Isolate tests with external server dependencies

Open
#3,624 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/tests component/internal-tools kind/test-flake lifecycle/frozen priority/P2
Dominant language
Go
Stars
8.7k
Forks
4.8k
Avg merge
4d 10h
Merged PRs (30d)
53

Description

Several tests at several layers (unit, integration, test-cmd.sh, etc) have dependencies on external servers like dockerhub, etc. When these servers have issues, they can block the merge queue.

Possible failures (all from https://github.com/openshift/origin/pull/3619):

--- FAIL: TestImage (0.91s)
    client_test.go:105: unexpected error: error getting X-Docker-Token from openshift/origin-not-found: Get http://index.docker.io/v1/repositories/openshift/origin-not-found/images/: EOF
error: no image or template matched "mysql"
!!! Error in hack/test-cmd.sh:501
  '[ "$(oc new-app mysql --name=db | grep db)" ]' exited with status 1
Call stack:
  1: hack/test-cmd.sh:501 main(...)
Exiting with status 1
--- FAIL: TestRunAll (2.03s)
    newapp_test.go:686: emptyDir volumes: Error mismatch! Expected <nil>, got no image or template matched "mysql": the image "ee2cb077006e3385648619d6adf66d5ac53cfb11c4b2708636c5be0a12b23ff7" in repository "library/mysql" was not found and may have been deleted

Our core tests should be able to run with minimal dependencies on external servers. Tests that depend on other servers should be moved to an extended test suite.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining client_test.go, hack/test-cmd.sh around line 501, and newapp_test.go around TestRunAll and line 686. Reproduce the Docker Hub and mysql failures, then identify which external-server-dependent tests belong in an extended suite while keeping core tests runnable with minimal external dependencies.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
ci-cd, testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.