compiler-explorer / compiler-explorer/infra
Another update failure
- Dominant language
- Python
- Stars
- 434
- Forks
- 429
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 38
Description
```
INFO:ce:Waiting for i-0dd4cc1be80c1b8a8@3.86.38.44 to reach autoscale lifecycle 'Standby'...
INFO:ce:...done
INFO:ce:Restarting service on i-0dd4cc1be80c1b8a8@3.86.38.44
INFO:ce:Waiting for instance to be Online i-0dd4cc1be80c1b8a8@3.86.38.44
Waiting...Ok, Everything is awesome!
INFO:ce:Moving i-0dd4cc1be80c1b8a8@3.86.38.44 out of standby
INFO:ce:Waiting for i-0dd4cc1be80c1b8a8@3.86.38.44 to reach autoscale lifecycle 'InService'...
ERROR:ce:Instance is not longer in an ASG: stopping
INFO:ce:Waiting for i-0dd4cc1be80c1b8a8@3.86.38.44 to reach ELB state 'healthy'...
Traceback (most recent call last):
File "", line 1, in
File "/home/ubuntu/infra/bin/lib/ce.py", line 942, in main
globals()[cmd + "_cmd"](kwargs)
File "/home/ubuntu/infra/bin/lib/ce.py", line 274, in instances_cmd
dispatch_global('instances', args)
File "/home/ubuntu/infra/bin/lib/ce.py", line 36, in dispatch_global
globals()['{}_{}_cmd'.format(sub, args['{}_sub'.format(sub)])](args)
File "/home/ubuntu/infra/bin/lib/ce.py", line 339, in instances_restart_cmd
restart_one_instance(as_group_name, instance, modified_groups)
File "/home/ubuntu/infra/bin/lib/ce.py", line 188, in restart_one_instance
wait_for_elb_state(instance, 'healthy')
File "/home/ubuntu/infra/bin/lib/ce.py", line 109, in wait_for_elb_state
instance.update()
File "/home/ubuntu/infra/bin/lib/instance.py", line 41, in update
exec_remote(self, ['sudo', 'systemctl', 'show', 'compiler-explorer']).split("\n") if
File "/home/ubuntu/infra/bin/lib/ssh.py", line 55, in exec_remote
return subprocess.check_output(ssh_args_for(command, instance)).decode('utf-8')
File "/home/ubuntu/infra/bin/lib/ssh.py", line 66, in ssh_args_for
'ubuntu@' + ssh_address_for(instance), '--'] + [f"'{c}'" for c in command]
TypeError: can only concatenate str (not "NoneType") to str
```
After:
```
Running in prod
Address Instance Id State Type ELB Service Version
184.72.68.82 i-0f62890a7f0852917 running c5.large healthy running 6618 (master)
35.172.215.142 i-00f3414a9076b9c95 running c5.large unhealthy running 6618 (master)
3.238.254.117 i-04a9dcdd078d5b08c running c5.large healthy running 6618 (master)
3.238.164.232 i-009b96487a7781a98 running c5.large healthy running 6618 (master)
3.84.169.206 i-010885afbe09ab66e running c5.large healthy running 6613 (master)
```
so, another case where we're not happy if a node goes south while updating.
Contributor guide
Assessment
This issue has not been assessed yet.