VR healthcheck wrong state
- Ngôn ngữ chính
- Java
- Star
- 3.1k
- Fork
- 1.4k
- Merge trung bình
- 6 ngày 19 giờ
- Pull request đã merge (30 ngày)
- 32
Mô tả
### problem
The listRouters API-call always returns healthcheckfailed false, even if the healthcheck fails.
Alert in CloudStack UI:
```
Health checks failed: 1 failing checks on router r-8-VM / c0c6a15f-3ad8-4936-ba6e-abe802ddba16
```
Error in EventQueue
```
{"eventDateTime":"2026-02-09 14:01:22 +0100","subject":"Health checks failed: 1 failing checks on router r-8-VM / c0c6a15f-3ad8-4936-ba6e-abe802ddba16","dataCenterId":"a8f95837-c05c-4a97-b100-53f87de8df24","event":"ALERT.SERVICE.DOMAINROUTER","body":"Health checks failed: 1 failing checks on router r-8-VM / c0c6a15f-3ad8-4936-ba6e-abe802ddba16","podId":"090ccb90-9411-4bab-bb0c-f5ab61e34ebb"}
{"eventDateTime":"2026-02-09 14:01:22 +0100","entityuuid":"c0c6a15f-3ad8-4936-ba6e-abe802ddba16","description":"Router r-8-VM / c0c6a15f-3ad8-4936-ba6e-abe802ddba16 has failing checks: router_version_check.py","event":"ROUTER.HEALTH.CHECKS","user":"c302beae-cb9d-11f0-a957-fa163ed04c4b","account":"c302a9b2-cb9d-11f0-a957-fa163ed04c4b","entity":"DomainRouter","status":"Completed"}
```
API call via cmk
```
(cloudstackmsu) 🐄 > list routers name=r-8-VM filter=name,hostname,healthchecksfailed,healthcheckresults,state, fetchhealthcheckresults=true listall=true
router:
count = 1
+--------+----------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+
| NAME | HOSTNAME | HEALTHCHECKSFAILED | HEALTHCHECKRESULTS | STATE |
+--------+----------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+
| r-8-VM | srv1-214 | false | [{"checkname":"connectivity.test","checktype":"basic","details":"Failed to fetch results with details: java.io.IOException: Stream closed\n\tat | Running |
| | | | java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)\n\tat java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)\n\tat | |
| | | | java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:287)\n\tat java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:330)\n\tat | |
| | | | java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:190)\n\tat java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)\n\tat | |
| | | | java.base/java.io.BufferedReader.fill(BufferedReader.java:162)\n\tat java.base/java.io.BufferedReader.readLine(BufferedReader.java:329)\n\tat | |
| | | | java.base/java.io.BufferedReader.readLine(BufferedReader.java:396)\n\tat com.cloud.utils.script.OutputInterpreter.processError(OutputInterpreter.java:41)\n\tat | |
| | | | com.cloud.utils.script.Script.execute(Script.java:336)\n\tat com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeInVR(LibvirtComputingResource.java:632)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeInVR(LibvirtComputingResource.java:620)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.checkRouterFileSystem(VirtualRoutingResource.java:431)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:403)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeQueryCommand(VirtualRoutingResource.java:239)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:130)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.wrapper.LibvirtNetworkElementCommandWrapper.execute(LibvirtNetworkElementCommandWrapper.java:35)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.wrapper.LibvirtNetworkElementCommandWrapper.execute(LibvirtNetworkElementCommandWrapper.java:29)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)\n\tat | |
| | | | com.cloud.agent.Agent.processRequest(Agent.java:813)\n\tat com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)\n\tat | |
| | | | com.cloud.utils.nio.Task.call(Task.java:83)\n\tat com.cloud.utils.nio.Task.call(Task.java:29)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat | |
| | | | java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat | |
| | | | java.base/java.lang.Thread.run(Thread.java:840)\n","lastupdated":"2026-02-11T13:01:46+0100","status":"FAILED","success":false},{"checkname":"filesystem.writable.test","checktype":"basic","details":"Failed | |
| | | | to fetch results with details: java.io.IOException: Stream closed\n\tat java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)\n\tat | |
| | | | java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)\n\tat java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:287)\n\tat | |
| | | | java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:330)\n\tat java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:190)\n\tat | |
| | | | java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)\n\tat java.base/java.io.BufferedReader.fill(BufferedReader.java:162)\n\tat | |
| | | | java.base/java.io.BufferedReader.readLine(BufferedReader.java:329)\n\tat java.base/java.io.BufferedReader.readLine(BufferedReader.java:396)\n\tat | |
| | | | com.cloud.utils.script.OutputInterpreter.processError(OutputInterpreter.java:41)\n\tat com.cloud.utils.script.Script.execute(Script.java:336)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeInVR(LibvirtComputingResource.java:632)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeInVR(LibvirtComputingResource.java:620)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.checkRouterFileSystem(VirtualRoutingResource.java:431)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:403)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeQueryCommand(VirtualRoutingResource.java:239)\n\tat | |
| | | | com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:130)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.wrapper.LibvirtNetworkElementCommandWrapper.execute(LibvirtNetworkElementCommandWrapper.java:35)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.wrapper.LibvirtNetworkElementCommandWrapper.execute(LibvirtNetworkElementCommandWrapper.java:29)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78)\n\tat | |
| | | | com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:2280)\n\tat | |
| | | | com.cloud.agent.Agent.processRequest(Agent.java:813)\n\tat com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1295)\n\tat | |
| | | | com.cloud.utils.nio.Task.call(Task.java:83)\n\tat com.cloud.utils.nio.Task.call(Task.java:29)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat | |
| | | | java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat | |
| | | | java.base/java.lang.Thread.run(Thread.java:840)\n","lastupdated":"2026-02-11T13:01:46+0100","status":"UNKNOWN","success":false}] | |
+--------+----------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+
```
Expected result: healthcheckfailed=true instead of healthcheckfailed=false
### versions
ACS 4.22.0
### The steps to reproduce the bug
1. ssh into one VR VM
2. shutdown the interfaces e.g. `ifdown eth0 ; ifdown eth1`
3. Wait about 10 minutes
4. Check Alerts in the UI
5. Check `cmk list routers filter=name,hostname,healthchecksfailed,healthcheckresults,state, fetchhealthcheckresults=true listall=true`
### What to do about it?
_No response_
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu tại VirtualRoutingResource.checkRouterFileSystem và theo dõi cách các kết quả health-check thất bại đi qua KVM agent vào phản hồi API listRouters. Tái hiện trường hợp bằng cách sử dụng fetchhealthcheckresults=true và so sánh healthchecksfailed với các kết quả FAILED được báo cáo; hoàn tất khi API báo cáo trạng thái thất bại nhất quán với các kết quả đó.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- java, python
- Lĩnh vực
- backend, infrastructure
- Loại issue
- Lỗi
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức độ hoạt động
- Đình trệ
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 35/100