apache / apache/cloudstack

Management server hangs when js.interpretation.enabled=true

Open
#12,523 16 comments 0 reactions 0 assignees Claimed by @weizhouapache View on GitHub
component:management-server Severity:Minor type:bug
Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
6d 19h
Merged PRs (30d)
32

Description

### problem

Enabling the global setting `js.interpretation.enabled=true` causes the CloudStack management server to hang during startup at the module loading phase. This prevents using the secondary storage selector feature (`createSecondaryStorageSelector`, `updateSecondaryStorageSelector` APIs).

### versions

- 4.20 branch (4.20.3.0-SNAPSHOT) - **AFFECTED** - needs fix before 4.20.3.0 release
- 4.20.2.0 (released) - Not affected (commit not present, setting does not exist, APIs work unconditionally)

Introduced by commit `03a4b9f` ("server,utils: improve js interpretation functionality") merged on Oct 16, 2025 into the 4.20 branch.

The commit:
1. Added `js.interpretation.enabled` as a hidden ConfigKey (default: false)
2. Made `CreateSecondaryStorageSelectorCmd` and `UpdateSecondaryStorageSelectorCmd` conditional on this setting
3. Made significant changes to `JsInterpreter.java` (security hardening, daemon workers, etc.)

When the setting is enabled, something in the initialization causes the MS server to hang.

### The steps to reproduce the bug

1. Deploy CloudStack 4.20.3.0
2. Verify server is running:
```
curl -s "http://localhost:8080/client/api?command=listZones" | head -5
```
3. Enable the js.interpretation.enabled setting:
```
mysql -u root -p cloud -e "UPDATE configuration SET value='true' WHERE name='js.interpretation.enabled';"
```
4. Restart management server:
```
systemctl restart cloudstack-management
```
5. Wait 3+ minutes and check server status:
```
curl -s "http://localhost:8080/client/api?command=listZones" | head -5
```

### Expected Result
Server starts successfully and `createSecondaryStorageSelector` API is available.

### Actual Result
Server returns 503 Service Unavailable. Log shows server stuck at module loading phase:
```
2026-01-26 19:53:01,937 DEBUG [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) Trying to obtain module [redfish] context.
2026-01-26 19:53:01,937 WARN [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) Application context not found for module definition [redfish]
```

Log timestamp remains static while current time progresses - server is hung.

#### Evidence

**4.20 branch build (shapeblue18197) - Bug present:**
```
[root@ref-trl-10730-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# rpm -qa | grep cloudstack
cloudstack-common-4.20.3.0-shapeblue18197.noarch
cloudstack-management-4.20.3.0-shapeblue18197.noarch

[root@ref-trl-10730-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# mysql -u root -p cloud -e "UPDATE configuration SET value='true' WHERE name='js.interpretation.enabled';"
[root@ref-trl-10730-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# systemctl restart cloudstack-management
[root@ref-trl-10730-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# sleep 180
[root@ref-trl-10730-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# curl -s "http://localhost:8080/client/api?command=listZones" | head -5

Error 503 Service Unavailable

```

**4.20.2.0 released version (shapeblue17631) - Works correctly:**
```
[root@ref-trl-10714-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# rpm -qa | grep cloudstack
cloudstack-common-4.20.2.0-shapeblue17631.noarch
cloudstack-management-4.20.2.0-shapeblue17631.noarch

mysql> SELECT name, value FROM configuration WHERE name LIKE '%interpret%';
Empty set (0.00 sec)

[root@ref-trl-10714-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# cmk list apis 2>/dev/null | grep -i "name.*selector"
"name": "createSecondaryStorageSelector",
"name": "updateSecondaryStorageSelector",
"name": "listSecondaryStorageSelectors",
"name": "removeSecondaryStorageSelector",
```

#### Impact
- Secondary storage selector feature is unusable on current 4.20 branch
- Quota tariff activation rules may also be affected (uses same JsInterpreter)
- Blocks testing of PR #10957 and any other functionality that depends on this feature

#### Related
- Commit: https://github.com/apache/cloudstack/commit/03a4b9f
- Blocks PR: #10957

### What to do about it?

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the startup hang with js.interpretation.enabled=true and inspect the management-server logs around DefaultModuleDefinitionSet. Read JsInterpreter.java and the changes from commit 03a4b9f, including CreateSecondaryStorageSelectorCmd and UpdateSecondaryStorageSelectorCmd, while checking the related PRs. Done means the management server starts normally with the setting enabled and the secondary storage selector APIs are available.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.