apache / apache/cloudstack

Retrying host maintenance falsely marks already-migrated VMs as stopped

Open
#14,026 2 comments 0 reactions 0 assignees View on GitHub
component:management-server type:bug
Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
6d 19h
Merged PRs (30d)
32

Description

CLOUDSTACK VERSION
4.22.1.1, KVM hypervisor

SUMMARY
Putting a host into maintenance moves its VMs off one at a time, which
can take a while. Requesting maintenance again on the same host before
the first request finishes causes it to retry moving every VM still
listed against that host, including ones already moved. That retry
fails as expected, but for some VMs it confuses CloudStack into
thinking the VM was powered off, and it gets marked stopped even though
it was never touched and kept running the whole time.

EXPECTED
Requesting maintenance again should not retry VMs already moved.

ACTUAL
Some already-moved VMs get falsely marked stopped.

NOTE
The false "stopped" status seems to come from CloudStack losing track
of which VMs are on which host. That could plausibly happen from causes
other than this one, but we only have evidence of it via this scenario.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing repeated maintenance requests on a KVM host in CloudStack 4.22.1.1, then trace the host-maintenance VM migration flow and how the host's VM list is updated. Done means a second request skips VMs already migrated, and those VMs remain running rather than being marked stopped.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.