Keystore Password Exposure in KVM Agent Security Setup

## Summary

A critical security vulnerability has been identified in the Apache CloudStack KVM agent security setup process where **randomly generated keystore passwords are logged in plaintext**. During the `LibvirtServerDiscoverer.setupAgentSecurity()` method execution, sensitive keystore passwords are exposed through SSH command logging in `SSHCmdHelper.sshExecuteCmdOneShot()`, compromising the security of agent-to-management-server communications.

## Vulnerability Details

**Component:** `com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer`
- `setupAgentSecurity()`
- `SSHCmdHelper.sshExecuteCmdWithResult()`
- `SSHCmdHelper.sshExecuteCmdOneShot()`

**Vulnerability Type:** Sensitive Information Disclosure / Password Exposure in Debug Logs (CWE-532, CWE-532)

## Technical Description

### Vulnerability Flow

#### 1. Password Generation and Password Exposure

The same password is used again in the certificate import command:

##### Expose source 1
```java
// void com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer.setupAgentSecurity(Connection sshConnection, String agentIp, String agentHostname)
final SSHCmdHelper.SSHCmdResult setupCertResult = SSHCmdHelper.sshExecuteCmdWithResult(sshConnection,
String.format("sudo /usr/share/cloudstack-common/scripts/util/%s " +
"/etc/cloudstack/agent/agent.properties %s " + // ← keystorePassword exposed again
"/etc/cloudstack/agent/%s %s " +
"/etc/cloudstack/agent/%s \"%s\" " +
"/etc/cloudstack/agent/%s \"%s\" " +
"/etc/cloudstack/agent/%s \"%s\"",
KeyStoreUtils.KS_IMPORT_SCRIPT,
keystorePassword, // Password in command
KeyStoreUtils.KS_FILENAME,
KeyStoreUtils.SSH_MODE,
KeyStoreUtils.CERT_FILENAME,
certificateCommand.getEncodedCertificate(),
KeyStoreUtils.CACERT_FILENAME,
certificateCommand.getEncodedCaCertificates(),
KeyStoreUtils.PKEY_FILENAME,
certificateCommand.getEncodedPrivateKey()));
```
##### Expose source 2

```java
// Answer com.cloud.baremetal.networkservice.BaremetalPingPxeResource.execute(PrepareCreateTemplateCommand cmd)
protected Answer execute(PrepareCreateTemplateCommand cmd) {
...

String script =
String.format("python /usr/bin/prepare_tftp_bootfile.py backup %1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s %11$s", _tftpDir, cmd.getMac(),
_storageServer, _share, _dir, cmd.getTemplate(), _cifsUserName, _cifsPassword, cmd.getIp(), cmd.getNetMask(), cmd.getGateWay()); // Password in command
if (!SSHCmdHelper.sshExecuteCmd(sshConnection, script)) {
return new Answer(cmd, false, "prepare for creating template failed, command:" + script);
}
...
}
```

##### Expose source 3

```java
// PreparePxeServerAnswer com.cloud.baremetal.networkservice.BaremetalPingPxeResource.execute(PreparePxeServerCommand cmd)
protected PreparePxeServerAnswer execute(PreparePxeServerCommand cmd) {

String script =
String.format("python /usr/bin/prepare_tftp_bootfile.py restore %1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s %11$s", _tftpDir, cmd.getMac(),
_storageServer, _share, _dir, cmd.getTemplate(), _cifsUserName, _cifsPassword, cmd.getIp(), cmd.getNetMask(), cmd.getGateWay()); // Password in command
if (!SSHCmdHelper.sshExecuteCmd(sshConnection, script)) {
return new PreparePxeServerAnswer(cmd, "prepare PING at " + _ip + " failed, command:" + script);
}
```

#### 2. Debug Logging Exposure

Both commands are logged in `SSHCmdHelper.sshExecuteCmdOneShot()`:

```java
public static SSHCmdResult sshExecuteCmdOneShot(com.trilead.ssh2.Connection sshConnection, String cmd) throws SshException {
LOGGER.debug("Executing cmd: " + cmd.split(KeyStoreUtils.KS_FILENAME)[0]); // ← Password logged here
Session sshSession = null;
try {
sshSession = sshConnection.openSession();
Thread.sleep(1000);

if (sshSession == null) {
throw new SshException("Cannot open ssh session");
}
// ... execution continues
}
...
if (!StringUtils.isAllEmpty(result.getStdOut(), result.getStdErr())) {
LOGGER.debug("SSH command: " + cmd.split(KeyStoreUtils.KS_FILENAME)[0] + "\nSSH command output:" + result.getStdOut().split("-----BEGIN")[0] + "\n" + result.getStdErr()); // ← Password logged here
}
}
```

### Incomplete Sanitization Attempt

The code attempts to sanitize by splitting on `KS_FILENAME` (likely `"agent.jks"`):

```java
cmd.split(KeyStoreUtils.KS_FILENAME)[0]
```

**However, this approach is fundamentally flawed:**

The password appears **before** `agent.jks` in the command string, so it's included in the first split segment and **fully exposed in logs**.

#### For example: Expose source 1 Command:
```bash
# Original command:
sudo .../keystore-import.sh /etc/.../agent.properties Xy9$mK2pL#4nQ7vR /etc/.../agent.jks ssh ...

# After split on "agent.jks" and taking [0]:
sudo .../keystore-import.sh /etc/.../agent.properties Xy9$mK2pL#4nQ7vR /etc/.../
# ❌ Password is BEFORE the split point, so it's INCLUDED in the logged output
```

Again, the password appears **before** the split point and is **fully exposed**.

## Root Cause Analysis

1. **Flawed Sanitization Logic**: The `split(KeyStoreUtils.KS_FILENAME)[0]` approach assumes the password appears *after* the keystore filename, but the actual command structure places it *before*

2. **Position-Dependent Vulnerability**: The sanitization logic is position-dependent but uses the wrong position marker

3. **No Pattern-Based Redaction**: No regex or pattern matching to identify and mask password parameters

4. **Debug Logging in Production**: Debug-level logging may be enabled in production environments, exposing sensitive data

Start with LibvirtServerDiscoverer.setupAgentSecurity() and SSHCmdHelper.sshExecuteCmdWithResult()/sshExecuteCmdOneShot(), then trace the command strings passed to the SSH helper. Review the bare-metal command construction examples as additional exposure paths. Done means sensitive passwords no longer appear in debug command or output logging, including the keystore setup flow.

Written by the indexing model from the issue text.