cephfs: Add KMIP support to CephFS fscrypt encryption
- Dominant language
- Go
- Stars
- 1.6k
- Forks
- 617
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 43
Description
Hello,
CephFS encryption with KMIP as the KMS backend doesn't work. I'd like to add this support and am opening this issue to get community input on which of two design directions to take: either let KMIP keep using envelope encryption (where the actual encryption key is stored in the volume itself and unwrapped by KMIP on demand), or have KMIP store the encryption key directly (similar to how the existing Vault integration works). Before writing code I'd like to confirm my understanding of the problem and hear which approach the maintainers prefer.
## What is the value to the end user? (why is it a priority?)
KMIP (Key Management Interoperability Protocol) is a vendor-neutral standard for applications to interact with key management systems, and is the interface most HSM appliances expose. ceph-csi already supports KMIP for RBD volume encryption, but the same KMS cannot back CephFS encryption today. Users whose only key management option is a KMIP-fronted HSM are therefore forced to either drop CephFS encryption or run HashiCorp Vault as a KMIP passthrough.
## Context
All KMS implementations in ceph-csi are either of the type `DEKStoreIntegrated` or `DEKStoreMetadata`. The following code comments explain what these types mean.
```go
// ceph-csi internal/kms/kms.go
// DEKStoreType describes what DEKStore needs to be configured when using a
// particular KMS. A KMS might support different DEKStores depending on its
// configuration.
type DEKStoreType string
const (
// DEKStoreIntegrated indicates that the KMS itself supports storing
// DEKs.
DEKStoreIntegrated = DEKStoreType("")
// DEKStoreMetadata indicates that the KMS should be configured to
// store the DEK in the metadata of the volume.
DEKStoreMetadata = DEKStoreType("metadata")
)
```
## Is-State
As you can see in the following code snippet the `ConfigureEncryption()` function anticipates that some KMS implementations do not implement the `GetSecret()` function which is needed to use fscrypt (kernel module which implements file-level encryption — block storage equivalent is dmcrypt).
```go
// ceph-csi internal/cephfs/store/volumeoptions.go
// ConfigureEncryption initializes the Ceph CSI key management from
// kmsID and credentials. Sets vo.Encryption on success.
func (vo *VolumeOptions) ConfigureEncryption(
ctx context.Context,
kmsID string,
credentials map[string]string,
) error {
kms, err := kmsapi.GetKMS(vo.Owner, kmsID, credentials)
if err != nil {
log.ErrorLog(ctx, "get KMS failed %+v: %v", vo, err)
return err
}
// With cephfs selecting the cipher is not possible
vo.Encryption, err = util.NewVolumeEncryption(kmsID, kms, nil)
if errors.Is(err, util.ErrDEKStoreNeeded) {
// fscrypt uses secrets directly from the KMS.
// Therefore we do not support an additional DEK
// store. Since not all "metadata" KMS support
// GetSecret, test for support here. Postpone any
// other error handling
_, err := vo.Encryption.KMS.GetSecret(ctx, "")
if errors.Is(err, kmsapi.ErrGetSecretUnsupported) {
return err
}
}
return nil
}
```
When we look for the KMIP KMS implementation we can clearly see that the `GetSecret()` function is not implemented hence the `ConfigureEncryption()` function will fail if KMIP is the KMS backend for ceph-csi cephfs encryption.
```go
// ceph-csi internal/kms/kmip.go
func (kms *kmipKMS) RequiresDEKStore() DEKStoreType {
return DEKStoreMetadata
}
func (kms *kmipKMS) GetSecret(ctx context.Context, volumeID string) (string, error) {
return "", ErrGetSecretUnsupported
}
```
The `kmipKMS` struct is of the `DekStoreType` `DEKStoreMetadata` which "indicates that the KMS should be configured to store the DEK in the metadata of the volume." However `kmipKMS` never actually needs to retrieve the secret via `GetSecret()`, instead getting the secret is done via the `DecryptDEK()` implementation as seen in the following code snippet.
```go
// ceph-csi internal/kms/kmip.go
// DecryptDEK calls either GET or DECRYPT operation on the KMIP kms to decrypt the DEK.
// The operation to call depends upon the config value `USE_CRYPTO_RPC`.
//
// If USE_CRYPTO_RPC == false : GET operation is used.
// If USE_CRYPTO_RPC == true : DECRYPT operation is used.
func (kms *kmipKMS) DecryptDEK(ctx context.Context, volumeID, encryptedDEK string) (string, error) {
if kms.useCryptoRPC {
return kms.decryptDEKUsingDecryptRPC(ctx, volumeID, encryptedDEK)
}
return kms.decryptDEKUsingRemoteKey(ctx, volumeID, encryptedDEK)
}
```
Thus the `GetSecret()` function did not need to be implemented to retrieve the secret because the `DecryptDEK()` retrieves the secret and decrypts it in one go.
## Should-State
Practically there are two ways KMIP could work in concert with fscrypt. The following code snippet shows the core interaction between the ceph-csi fscrypt logic and the KMS backend.
```go
// ceph-csi internal/util/fscrypt/fscrypt.go
switch encryption.KMS.RequiresDEKStore() {
case kms.DEKStoreIntegrated:
passphrase, err = encryption.GetCryptoPassphrase(ctx, volID)
case kms.DEKStoreMetadata:
passphrase, err = encryption.KMS.GetSecret(ctx, volID)
}
```
So to make KMIP work with ceph-csi fscrypt, one of two things needs to happen:
1. Keep KMIP as a `DEKStoreMetadata` and implement its `GetSecret()` method, or
2. Add a new (or extended) KMIP struct of type `DEKStoreIntegrated` which can use `GetCryptoPassphrase()` (analogous to how the `vaultKMS` implementation works).
## Questions
I understand that from an fscrypt perspective there is no KEK, DEK (envelope encryption) needed. The KMS system just needs to store a key which can directly be used by fscrypt. This would correspond to the `DEKStoreIntegrated` semantics.
1. **Do we want to implement this key model for ceph-csi fscrypt with KMIP** — i.e. implement a new KMIP struct that is a `DEKStoreIntegrated` key store?
2. **Alternatively, we could store the fscrypt DEK as an encrypted DEK on cephfs metadata and decrypt it in the KMIP KMS.** This would match the current `DEKStoreType` the KMIP struct implements (`DEKStoreMetadata`). Would this be a better approach?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading internal/cephfs/store/volumeoptions.go, internal/kms/kmip.go, and internal/util/fscrypt/fscrypt.go, focusing on ConfigureEncryption(), GetSecret(), DecryptDEK(), and the DEKStoreType switch. Compare the two proposed KMIP approaches with the existing Vault integration and seek maintainer guidance; done means one design is approved with a scoped implementation path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100