apache / apache/cloudstack

Cloudstack 4.20 UI becomes irresponsive. Restart the server to recover, it happens every 10 days.

未关闭
#10,578 24 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
status:needs-reproducing
主要语言
Java
星标
3.1k
派生
1.4k
平均合并
6 天 19 小时
30 天内合并 PR
32

描述

Description:
I'm experiencing a critical issue with CloudStack 4.20 where all API endpoints become unresponsive approximately every 10 days. The only temporary resolution is to restart the CloudStack management server.

Observed Behavior:

API requests timeout/fail completely after ~10 days of uptime

No explicit ERROR messages in logs prior to outage

Found an unusually large INFO-level log entry (3MB per line) that might be relevant

Attached log file: [filename.log] (Please ensure you actually attach the file via GitHub interface)

Environment:

CloudStack Version: 4.20.0.0

OS:Ubuntu 24.04

Steps to Reproduce:

Start CloudStack management server

Operate normally for ~10 days

API services become unavailable without obvious triggers

Expected Behavior:
API endpoints should remain available continuously without requiring manual restarts.

Additional Context:

The large INFO-level log entry repeats periodically (full content attached)

No observed resource exhaustion (CPU/MEM) before outages

Problem persists across multiple maintenance windows

Troubleshooting Attempted:

Reviewed standard error logs - no smoking gun

Monitored system resources - no apparent bottlenecks

Server restart temporarily resolves the issue

Request:
Please help investigate:

Potential memory leaks or thread blocking in the 4.20 codebase

Significance of the oversized INFO log entries

Update to Original Issue:
Further analysis of the oversized INFO log reveals repetitive entries related to createVPCOffering API calls. The JSON payload in these logs appears to be abnormally large (3MB per line) and contains repetitive configuration data.

Key Log Excerpt Pattern:
INFO [c.c.a.ApiServlet] (qtp123456789-42:) {cmd="createVPCOffering", ... JSON payload (3MB) ...}

![Image](https://github.com/user-attachments/assets/f900807c-3aca-4028-bea7-ea76cc1c94d2)

贡献指南

打开贡献指南

调研方向

首先检查附加的日志和重复出现的 createVPCOffering 条目,重点关注 3MB INFO payload 模式。然后检查由 ApiServlet 表示的 CloudStack 4.20 API 路径,并在可能的情况下重现大约 10 天的中断;完成的标准是确定原因、定义受影响的代码更改,并确认无需重启即可继续提供 API 请求。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
api, backend, cloud, infrastructure
Issue 类型
缺陷
难度
5/5
预计耗时
一周以上
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。