cloud-bulldozer / cloud-bulldozer/perfscale-managed-services
Controlling installation result of osde2e with exit code is not enough
- Dominant language
- Python
- Stars
- 2
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
Installing with a wrong ocm token results on thread exiting with exit code 0
```
2021-03-10 10:51:26.081 INFO osde2e-wrapper - main: Attempting to start 10 clusters with 2 batch size
2021-03-10 10:51:26.089 INFO osde2e-wrapper - _build_cluster: Started cluster osde2e-o3z-0001 (1 of 10)
2021-03-10 10:51:26.092 INFO osde2e-wrapper - _build_cluster: Started cluster osde2e-o3z-0002 (2 of 10)
2021-03-10 10:51:26.544 INFO osde2e-wrapper - _watcher: Requested Clusters for test 22b46690-9244-4a5d-935e-3c48f27a6338: 0
2021-03-10 10:51:31.888 INFO osde2e-wrapper - _build_cluster: Attempting to load metadata json
2021-03-10 10:51:31.888 ERROR osde2e-wrapper - _build_cluster: [Errno 2] No such file or directory: '/tmp/22b46690-9244-4a5d-935e-3c48f27a6338/osde2e-o3z-0002/metadata.json'
2021-03-10 10:51:31.889 ERROR osde2e-wrapper - _build_cluster: Failed to load metadata.json file located /tmp/22b46690-9244-4a5d-935e-3c48f27a6338/osde2e-o3z-0002
2021-03-10 10:51:31.889 INFO osde2e-wrapper - _download_kubeconfig: Attempting to load metadata json
2021-03-10 10:51:31.890 ERROR osde2e-wrapper - _download_kubeconfig: Failed to load cluster-id from metadata.json file located on /tmp/22b46690-9244-4a5d-935e-3c48f27a6338/osde2e-o3z-0002, kubeconfig file wont be downloaded
```
Installation logs:
```
2021/03/10 10:51:26 addon timeout is 30
2021/03/10 10:51:26 load.go:57: Custom YAML config provided, loading from cluster_account.yaml
2021/03/10 10:51:26 e2e.go:260: Could not create reporter directory: mkdir /tmp/22b46690-9244-4a5d-935e-3c48f27a6338/osde2e-o3z-0001: file exists
2021/03/10 10:51:26 e2e.go:303: Outputting log to build log at /tmp/22b46690-9244-4a5d-935e-3c48f27a6338/osde2e-o3z-0001/test_output.log
2021/03/10 10:51:26 versions.go:34: Querying cluster versions endpoint.
2021/03/10 10:51:26 go_logger.go:152: OCM auth: failed to get tokens, got http code 400, will not attempt to retry. err: invalid_grant: Offline user session not found
2021/03/10 10:51:26 retryer.go:23: error during OCM attempt: can't get access token: invalid_grant: Offline user session not found
2021/03/10 10:51:30 go_logger.go:152: OCM auth: failed to get tokens, got http code 400, will not attempt to retry. err: invalid_grant: Offline user session not found
2021/03/10 10:51:30 retryer.go:23: error during OCM attempt: can't get access token: invalid_grant: Offline user session not found
2021/03/10 10:51:31 go_logger.go:152: OCM auth: failed to get tokens, got http code 400, will not attempt to retry. err: invalid_grant: Offline user session not found
2021/03/10 10:51:31 retryer.go:23: error during OCM attempt: can't get access token: invalid_grant: Offline user session not found
```
[This conditional](https://github.com/cloud-bulldozer/perfscale-managed-services/blob/803e35a0f196d317b25e6d2288e1fbb5980d6edf/osde2e/osde2e-wrapper.py#L131) is not being executed, as we dont see the "Failed to build cluster" logging.
We need to fix the failed cluster installation detection but also assigning a default cluster-id as "not generated" or something if we detect that metadata.json file is not generated or generated without cluster-id
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in osde2e/osde2e-wrapper.py at the linked conditional around line 131, then trace how installation results and metadata.json are handled. Reproduce the wrong-token case using the logged failure, and verify that failed cluster installation is detected and logged while a missing or incomplete metadata.json receives a default cluster-id such as "not generated".
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100