apache / apache/cloudstack

Plaintext VM User-Data and SSH Public Key Log Exposure in Baremetal Kickstart PXE Plugin

Open
#13,307 0 comments 0 reactions 0 assignees View on GitHub
component:logging type:technical-debt
Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
6d 19h
Merged PRs (30d)
32

Description

### Advisory Details

**Title**: Plaintext VM User-Data and SSH Public Key Log Exposure in Baremetal Kickstart PXE Plugin

**Description**:

A critical sensitive information leak vulnerability (CWE-532) exists in the Apache CloudStack Baremetal Kickstart PXE plugin. When orchestrating virtual instances on baremetal hosts via PXE booting, the management server communicates tenant VM metadata (including custom user-data and SSH public keys) by executing command-line scripts via SSH on the PXE server node.

Due to a flawed sanitization logic in `SSHCmdHelper.java`'s `sshExecuteCmdOneShot()` execution, which only attempts to mask or truncate commands by splitting on the keystore file token `"cloud.jks"` (`KeyStoreUtils.KS_FILENAME`), all command strings lacking `"cloud.jks"` are written fully unmasked and exposed in system debug logs. As a result, standard tenant initialization passwords, keys, and SSH public keys are logged in plaintext, allowing any user or process with system debug log access to compromise tenant virtualization environments.

### Summary

An incomplete logging sanitization mechanism in the Baremetal Kickstart PXE resource flow allows plaintext sensitive VM initialization user-data and SSH public keys to be exposed in standard system debug logs, compromising client environment credentials.

### Details

#### Root Cause Analysis

When deploying or starting virtual machines in a baremetal environment, the `VmDataCommand` is dispatched. The execution flow is traced as follows:
```
[User API Client (deployVirtualMachine / updateVirtualMachine)]
--> [UserVmManagerImpl / CommandSetupHelper.generateVmDataCommand]
--> [VmDataCommand DTO (userdata & public keys)]
--> [AgentManagerImpl.send / BaremetalKickStartPxeResource.execute]
```

Within `plugins/hypervisors/baremetal/src/main/java/com/cloud/baremetal/networkservice/BaremetalKickStartPxeResource.java`'s `execute(VmDataCommand cmd)` method:
```java
private Answer execute(VmDataCommand cmd) {
com.trilead.ssh2.Connection sshConnection = new com.trilead.ssh2.Connection(_ip, 22);
try {
List vmData = cmd.getVmData();
StringBuilder sb = new StringBuilder();
for (String[] data : vmData) {
String folder = data[0];
String file = data[1];
String contents = (data[2] == null) ? "none" : data[2];
sb.append(cmd.getVmIpAddress());
sb.append(",");
sb.append(folder);
sb.append(",");
sb.append(file);
sb.append(",");
sb.append(contents);
sb.append(";");
}
String arg = StringUtils.stripEnd(sb.toString(), ";");

sshConnection.connect(null, 60000, 60000);
if (!sshConnection.authenticateWithPassword(_username, _password)) {
logger.debug("SSH Failed to authenticate with user {} credentials", _username);
throw new ConfigurationException(String.format("Cannot connect to PING PXE server(IP=%1$s, username=%2$s", _ip, _username));
}

String script = String.format("python /usr/bin/baremetal_user_data.py '%s'", arg);
if (!SSHCmdHelper.sshExecuteCmd(sshConnection, script)) {
return new Answer(cmd, false, "Failed to add user data, command:" + script);
}
```

The constructed command resolves to:
`"python /usr/bin/baremetal_user_data.py '10.0.0.10,metadata,userdata,PlaintextSuperSecretPassword123;10.0.0.10,metadata,sshkey,ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuVariantPublicSSHKey'"`

This command string contains the plaintext VM user-data and SSH public keys.

Inside `utils/src/main/java/com/cloud/utils/ssh/SSHCmdHelper.java`'s `sshExecuteCmdOneShot()` execution:
```java
public static SSHCmdResult sshExecuteCmdOneShot(com.trilead.ssh2.Connection sshConnection, String cmd) throws SshException {
LOGGER.debug("Executing cmd: " + cmd.split(KeyStoreUtils.KS_FILENAME)[0]);
...
if (!StringUtils.isAllEmpty(result.getStdOut(), result.getStdErr())) {
LOGGER.debug("SSH command: " + cmd.split(KeyStoreUtils.KS_FILENAME)[0] + ...);
}
}
```

Because `"cloud.jks"` (`KeyStoreUtils.KS_FILENAME`) is not present in the Python script execution, the split operation fails to truncate any portion of the command, logging the entire command—and thus the sensitive user-data and keys—directly to standard system debug logs (e.g., `vmops.log` or `management-server.log`).

### PoC

#### Prerequisites

* Target CloudStack setup must have the Baremetal Kickstart PXE plugin configured.
* System debug log output enabled.

#### Reproduction Steps

1. Set up the isolated laboratory environment:
Download: [docker-compose.yml](https://gist.github.com/YLChen-007/295622493e4f303103755f232ff5dc20)
Run: `docker compose up -d`

2. Download and run the defect verification script simulating Kickstart PXE metadata transmission command logging:
Download: [verification_test_Issue-cloudstack-12030.py](https://gist.github.com/YLChen-007/001a9614e0f3daffe6ed420e8e62bfe2)
Run: `python3 verification_test_Issue-cloudstack-12030.py`

3. Download and run the scientific control group script demonstrating that the filtering logic only works under standard keystore operations containing `"cloud.jks"`:
Download: [control-masked_output.py](https://gist.github.com/YLChen-007/bd4982b3ab321a7f09ee42ce071e94e8)
Run: `python3 control-masked_output.py`

### Log of Evidence

```
[*] Running Issue-cloudstack-12030-BaremetalKickstartPxe Sensitive Data Exposure Verification...
[*] Defect Verification - Input Tracing:
- Sensitive VM User-Data: PlaintextSuperSecretPassword123
- Sensitive SSH Public Key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuVariantPublicSSHKey

--- EXPERIMENT RESULTS (DEBUG LOGS) ---
[*] Logged VmDataCommand: python /usr/bin/baremetal_user_data.py '10.0.0.10,metadata,userdata,PlaintextSuperSecretPassword123;10.0.0.10,metadata,sshkey,ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuVariantPublicSSHKey'
---------------------------------------

[+] SUCCESS: Plaintext sensitive credentials leaked successfully in standard debug logs!
[+] Status: DEFECT-CONFIRMED
```

### Impact

* **Vulnerability Type**: CWE-532 (Insertion of Sensitive Information into Log File)
* **Impact**: Unauthenticated exposure of tenant environment initialization metadata, including plaintext custom user-data (passwords, access keys, script tokens) and SSH public keys. This allows system administrators or local attackers with log file read access to fully compromise tenant virtual machines and API interfaces.

### Affected products

- **Ecosystem**: maven
- **Package name**: org.apache.cloudstack:cloudstack
- **Affected versions**: <= 4.22.1.0
- **Patched versions**:

### Severity

- **Severity**: High
- **Vector string**: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N

### Weaknesses

- **CWE**: CWE-532: Insertion of Sensitive Information into Log File

### Occurrences

| Permalink | Description |
| :--- | :--- |
| [https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/utils/src/main/java/com/cloud/utils/ssh/SSHCmdHelper.java#L166-L168](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/utils/src/main/java/com/cloud/utils/ssh/SSHCmdHelper.java#L166-L168) | Flawed logging logic that relies on `KeyStoreUtils.KS_FILENAME` split-sanitization in `sshExecuteCmdOneShot`. |
| [https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/plugins/hypervisors/baremetal/src/main/java/com/cloud/baremetal/networkservice/BaremetalKickStartPxeResource.java#L111-L141](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/plugins/hypervisors/baremetal/src/main/java/com/cloud/baremetal/networkservice/BaremetalKickStartPxeResource.java#L111-L141) | Execution of `VmDataCommand` using command string containing raw metadata via `SSHCmdHelper.sshExecuteCmd`. |
| [https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/utils/src/main/java/com/cloud/utils/ssh/SSHCmdHelper.java#L228-L232](https://github.com/apache/cloudstack/blob/348ce953a99246a756b527994f7745a7be038234/utils/src/main/java/com/cloud/utils/ssh/SSHCmdHelper.java#L228-L232) | Flawed logging of the command execution output using `KS_FILENAME` split-sanitization. |

Contributor guide

Open the contributing guide

Research direction

Start with sshExecuteCmdOneShot() in utils/src/main/java/com/cloud/utils/ssh/SSHCmdHelper.java and execute(VmDataCommand cmd) in plugins/hypervisors/baremetal/src/main/java/com/cloud/baremetal/networkservice/BaremetalKickStartPxeResource.java. Run the provided verification script to trace the logged command and confirm that VM user-data and SSH keys are exposed. Done means sensitive metadata is no longer emitted in debug logs during this flow.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
infrastructure, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.