canonical / canonical/pycloudlib

start/stop operations are not atomic

Open
#369 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22
Forks
42
PR merge metrics
No merged PRs in 30d

Description

Calling `instance.wait_for_stop()` followed by `instance.start()` will sometimes result in the following exception:
```
> raise RuntimeError(errmsg)
E RuntimeError: Failure (rc=1): Error: The instance is already running
```
I think that lxc is reporting `STOPPED` prior to the image actually being (re)bootable. I've seen the same issue on occasion when manually doing a stop/start on images, but didn't realize that this was a problem for our integration tests until debugging a flaky integration test which does this.

In the observed case that I've seen this, the shutdown is initiated by the image. I don't know whether a pycloudlib-initiated shutdown will produce the same effect, but I think that it would, given the effect.

I've filed an issue against lxd, but until it gets fixed we could probably introduce a retry loop. I don't see this tested in the integration test, so we should probably add it too. The comment in that test "Test is unstable but most stable on lxd containers" leads me to suspect that other platforms are similarly affected.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.