cryostatio / cryostatio/cryostat-operator

Incorrect keystore password

Open
#266 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
37
Forks
22
Avg merge
21h 37m
Merged PRs (30d)
22

Description

I encountered this bug, where the keystore is created by cert-manager, but the password is incorrect:
```
INFO: Local save path for flight recordings set as /opt/cryostat.d/recordings.d
Exception in thread "main" io.vertx.core.VertxException: java.io.IOException: keystore password was incorrect
at io.vertx.core.net.impl.SSLHelper.createContext(SSLHelper.java:336)
at io.vertx.core.net.impl.SSLHelper.getContext(SSLHelper.java:511)
at io.vertx.core.net.impl.SSLHelper.validate(SSLHelper.java:536)
at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:284)
at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:205)
at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:186)
at io.cryostat.net.HttpServer.start(HttpServer.java:108)
at io.cryostat.Cryostat.main(Cryostat.java:77)
Caused by: java.io.IOException: keystore password was incorrect
at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2117)
at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:222)
at java.base/java.security.KeyStore.load(KeyStore.java:1479)
at io.vertx.core.net.impl.KeyStoreHelper.loadKeyStoreOptions(KeyStoreHelper.java:222)
at io.vertx.core.net.KeyStoreOptionsBase.getHelper(KeyStoreOptionsBase.java:148)
at io.vertx.core.net.KeyStoreOptionsBase.getKeyManagerFactory(KeyStoreOptionsBase.java:166)
at io.vertx.core.net.impl.SSLHelper.getKeyMgrFactory(SSLHelper.java:341)
at io.vertx.core.net.impl.SSLHelper.createContext(SSLHelper.java:294)
... 7 more
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
... 15 more
```

Looking at the secrets in my namespace, it appears that the TLS secrets created by cert-manager were left behind by an old deployment:
```
$ oc get secret
NAME TYPE DATA AGE
cryostat-operator-service-account-dockercfg-v5zb9 kubernetes.io/dockercfg 1 11m
cryostat-operator-service-account-token-6vfsv kubernetes.io/service-account-token 4 11m
cryostat-operator-service-account-token-9b2pn kubernetes.io/service-account-token 4 11m
cryostat-sample-ca kubernetes.io/tls 3 21h
cryostat-sample-dockercfg-vbvnn kubernetes.io/dockercfg 1 10m
cryostat-sample-grafana-basic Opaque 2 10m
cryostat-sample-grafana-tls kubernetes.io/tls 3 21h
cryostat-sample-jmx-auth Opaque 2 10m
cryostat-sample-keystore Opaque 1 10m
cryostat-sample-tls kubernetes.io/tls 4 21h
cryostat-sample-token-kgv4x kubernetes.io/service-account-token 4 10m
cryostat-sample-token-mxdlh kubernetes.io/service-account-token 4 10m
```

I suspect this was caused by setting `spec.enableCertManager` to false before the certificates became ready. This prevented the operator from setting the Cryostat CR as owner of the secrets.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the operator reconciliation path for the Cryostat CR and the spec.enableCertManager transition, using the reported TLS secrets and startup stack trace as evidence. Trace how cert-manager-created secrets and the keystore password are handled when certificates are not ready. Done means the reported deployment sequence no longer leaves Cryostat unable to start with an incorrect keystore password.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.