Dstack-TEE / Dstack-TEE/dstack
`bootstrap_keys()` auto-bootstrap path does not write `bootstrap-info.json`
- Dominant language
- Rust
- Stars
- 544
- Forks
- 96
- Avg merge
- 17h 57m
- Merged PRs (30d)
- 117
Description
When KMS is configured with `auto_bootstrap_domain` (non-empty string in `kms.toml`), the auto-bootstrap code path in `onboard_service.rs:bootstrap_keys()` generates keys and stores them via `keys.store(cfg)`, but **never writes `bootstrap-info.json`**.
This causes `GetMeta` to return `bootstrap_info: null`, which blocks the `kms:set-info` hardhat task from registering the KMS on-chain.
## Steps to Reproduce
1. Deploy KMS CVM with `auto_bootstrap_domain` set in `kms.toml`:
```toml
[onboard]
auto_bootstrap_domain = "kms.example.com"
quote_enabled = true
```
2. KMS starts, auto-bootstrap runs successfully (keys generated, certs created)
3. Query GetMeta:
```bash
curl -sk https://localhost:9100/prpc/KMS.GetMeta?json | jq .bootstrap_info
```
4. Result: `null`
## Root Cause
**Auto-bootstrap** (`onboard_service.rs:330-339`):
```rust
pub(crate) async fn bootstrap_keys(cfg: &KmsConfig) -> Result<()> {
let keys = Keys::generate(
&cfg.onboard.auto_bootstrap_domain,
cfg.onboard.quote_enabled,
)
.await
.context("Failed to generate keys")?;
keys.store(cfg)?; // Stores 8 files (keys, certs, rpc-domain) — NOT bootstrap-info.json
Ok(())
}
```
**Manual bootstrap** (`onboard_service.rs:54-78`):
```rust
async fn bootstrap(self, request: BootstrapRequest) -> Result {
let quote_enabled = self.state.config.onboard.quote_enabled;
let keys = Keys::generate(&request.domain, quote_enabled)
.await
.context("Failed to generate keys")?;
let k256_pubkey = keys.k256_key.verifying_key().to_sec1_bytes().to_vec();
let ca_pubkey = keys.ca_key.public_key_der();
let attestation = if quote_enabled {
Some(attest_keys(&ca_pubkey, &k256_pubkey).await?)
} else {
None
};
let cfg = &self.state.config;
let response = BootstrapResponse {
ca_pubkey,
k256_pubkey,
attestation: attestation.unwrap_or_default(),
};
// Store the bootstrap info
safe_write(cfg.bootstrap_info(), serde_json::to_vec(&response)?)?; // ← Writes the file
keys.store(cfg)?;
Ok(response)
}
```
The manual path extracts public keys, optionally generates an attestation quote via `attest_keys()`, constructs a `BootstrapResponse`, and writes it to `{cert_dir}/bootstrap-info.json` (defined in `config.rs:88-90` as `self.cert_dir.join("bootstrap-info.json")`). The auto-bootstrap path does none of this.
**`GetMeta`** (`main_service.rs:323-326`) silently returns `None` when the file is missing:
```rust
let bootstrap_info = fs::read_to_string(self.state.config.bootstrap_info())
.ok()
.and_then(|s| serde_json::from_str(&s).ok());
```
## Impact
- `GetMeta` returns `bootstrap_info: null`
- `kms:set-info` hardhat task fails (cannot read/parse null bootstrap info)
- KMS public keys and TDX attestation quote are never registered on-chain
- Applications cannot verify KMS attestation through the smart contract
- The entire on-chain trust anchor is missing
No workaround exists — `bootstrap-info.json` is only written in one place in the codebase (the manual `bootstrap()` RPC handler at line 75). No other service or background task populates it.
## History
- `ae75c86e` (2025-01-12): Added `bootstrap_keys()` — no `bootstrap-info.json` write
- `6900f54e` (2025-01-15): Added `bootstrap-info.json` write to the **manual** `bootstrap()` path only
The auto path has never written this file since it was introduced.
## Suggested Fix
Add public key extraction, attestation quote generation, and `bootstrap-info.json` writing to `bootstrap_keys()`, mirroring the manual `bootstrap()` path:
```rust
pub(crate) async fn bootstrap_keys(cfg: &KmsConfig) -> Result<()> {
let keys = Keys::generate(
&cfg.onboard.auto_bootstrap_domain,
cfg.onboard.quote_enabled,
)
.await
.context("Failed to generate keys")?;
keys.store(cfg)?;
// Write bootstrap-info.json (same as manual bootstrap path)
let k256_pubkey = keys.k256_key.verifying_key().to_sec1_bytes().to_vec();
let ca_pubkey = keys.ca_key.public_key_der();
let attestation = if cfg.onboard.quote_enabled {
Some(attest_keys(&ca_pubkey, &k256_pubkey).await?)
} else {
None
};
let response = BootstrapResponse {
ca_pubkey,
k256_pubkey,
attestation: attestation.unwrap_or_default(),
};
safe_write(cfg.bootstrap_info(), serde_json::to_vec(&response)?)?;
Ok(())
}
```
Note: The `attest_keys()` function (line 351) generates a TDX attestation quote binding both the P256 CA public key and the secp256k1 key. This is needed for on-chain registration to include a valid attestation.
### Also affected: `feat/auto-onboard-kms` branch
The `auto_onboard_keys()` function on the unmerged `feat/auto-onboard-kms` feature branch has the same gap — it generates keys and stores them but does not write `bootstrap-info.json`.
## Environment
- dstack version: v0.5.7
- Platform: TDX-enabled server (Intel Xeon with TDX support)
- Ubuntu 24.04 LTS
Contributor guide
Assessment
This issue has not been assessed yet.