apache / apache/cloudstack

CloudStack 4.22: `prepareHostForMaintenance` throws NPE when stale destroyed volume references removed storage pool

未關閉
#13,124 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Java
星號
3.1k
分支
1.4k
平均合併
6 天 19 小時
30 天內合併 PR
32

描述

### problem

## Description

While attempting to place a KVM host into Maintenance mode in CloudStack 4.22, the maintenance operation failed with a `NullPointerException`.

The issue appears to occur when a VM has a stale/destroyed volume entry in the `volumes` table that still references a removed storage pool.

Instead of gracefully ignoring the stale volume metadata or returning a user-facing validation error, CloudStack crashes during maintenance preparation.

## Environment

* CloudStack version: 4.22
* Hypervisor: KVM
* Primary storage: NFS
* Database: MySQL/MariaDB

## Steps to reproduce

1. Have a VM with:

* one valid active ROOT volume
* one stale/destroyed ROOT volume entry still present in the `volumes` table
2. The stale volume references a storage pool which has already been removed.
3. Attempt to place the host running the VM into Maintenance mode.

Example problematic DB state:

### VM

```sql
SELECT id, uuid, name, instance_id, pool_id, state, removed
FROM volumes
WHERE instance_id = 446
ORDER BY id;
```

Result:

```text
+------+--------------------------------------+----------+-------------+---------+---------+---------+
| id | uuid | name | instance_id | pool_id | state | removed |
+------+--------------------------------------+----------+-------------+---------+---------+---------+
| 928 | 6dea3d6f-bd6d-4e8b-9524-6e99c029694c | ROOT-446 | 446 | 4 | Destroy | NULL |
| 1554 | 80acf9ae-b047-41b8-bded-cdceb6de7051 | ROOT-446 | 446 | 2 | Ready | NULL |
+------+--------------------------------------+----------+-------------+---------+---------+---------+
```

### Storage pool state

The stale volume references storage pool ID 4, which is already removed:

```text
storage_pool_name: Export-Domain
storage_pool_removed: 2026-04-23 13:44:08
```

## Actual result

Host maintenance fails with:

```text
java.lang.NullPointerException: Cannot invoke
"org.apache.cloudstack.storage.datastore.db.StoragePoolVO.isLocal()"
because "storagePool" is null
```

Relevant stack trace:

```text
at com.cloud.vm.UserVmManagerImpl.isAnyVmVolumeUsingLocalStorage(UserVmManagerImpl.java:7558)
at com.cloud.vm.UserVmManagerImpl.isVMUsingLocalStorage(UserVmManagerImpl.java:7121)
at com.cloud.resource.ResourceManagerImpl.doMaintain(ResourceManagerImpl.java:1553)
at com.cloud.resource.ResourceManagerImpl.maintain(ResourceManagerImpl.java:1653)
at org.apache.cloudstack.api.command.admin.host.PrepareForHostMaintenanceCmd.execute(PrepareForHostMaintenanceCmd.java:99)
```

## Expected result

CloudStack should not throw an unhandled `NullPointerException`.

Possible expected behavior:

* ignore destroyed/removed stale volumes during maintenance evaluation
* skip volumes attached to removed pools
* or return a proper validation error identifying the problematic VM/volume

## Workaround

Marking the stale destroyed volume row as removed allowed maintenance to proceed:

```sql
UPDATE volumes
SET removed = NOW()
WHERE id = 928
AND instance_id = 446
AND state = 'Destroy'
AND removed IS NULL
AND pool_id = 4;
```

## Additional notes

The issue appears to be triggered specifically by:

* stale destroyed volume rows
* still linked to an active/running VM
* referencing removed storage pools
* while evaluating VM local-storage usage during host maintenance

### versions

The versions ACS 4.22, KVM (should not be relevant)

### The steps to reproduce the bug

1.
2.
3.
...

### What to do about it?

_No response_

貢獻指南

開啟貢獻指南

研究方向

從第 7558 行的 UserVmManagerImpl.isAnyVmVolumeUsingLocalStorage 開始,追蹤其在 isVMUsingLocalStorage 和 ResourceManagerImpl.doMaintain 中的呼叫端。重現已銷毀的過時磁碟區和已移除的儲存池狀態,然後確認維護不再引發未處理的 NullPointerException,並確認針對過時磁碟區所選擇的行為已由相關測試涵蓋。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
java, mariadb, mysql
領域
backend, cloud, infrastructure
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
冷清
描述清晰度
基本清楚
新手友好度
55/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。