ca: UpdateConfiguration fails if initialization is not complete
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 39
Description
This issue was reported by @lkysow from this [test failure](https://app.circleci.com/pipelines/github/hashicorp/consul-k8s/3002/workflows/02c23af9-df43-489d-b100-364ce3e9c090/jobs/21771).
The problem seems to be that while the CAManager "state machine" allows `UpdateConfiguration` to be called, it ends up failing [here](https://github.com/hashicorp/consul/blob/v1.10.3/agent/consul/leader_connect_ca.go#L841-L865) because there is no existing provider.
Before https://github.com/hashicorp/consul/pull/9570 this would not have been an issue because nothing would work until `InitializeCA` was complete.
It seems like we have two options to solve this:
1. Do not allow `UpdateConfiguration` until the provider is Initialized. This seems like the easiest option and would help reduce complexity in the `CAMananger`.
2. Allow `UpdateConfiguration` to save the config, but skip the other steps and let `InitializeCA` handle it when it runs.
I think there is a really good argument to be made for option 1. The failures in the test output are a great example. The test continually ran `CASetConfig` (which calls `UpdateConfiguration`) [in a loop](https://github.com/hashicorp/consul-k8s/blob/main/control-plane/subcommand/get-consul-client-ca/command_test.go#L255-L264). It would sleep for 25ms between iterations, but because `UpdateConfiguration` would run for 500ms before failing (see the log output), every `InitializeCA` attempt would still fail, because the "state" was `reconfig` every time. It spent 20x more time with the state in "reconfig" than in "uninitialized" (the state required by InitializeCA).
Related issue #10634, for making more general improvements to this state management.
Contributor guide
Research direction
Start with the CAManager state handling around UpdateConfiguration and InitializeCA, then inspect agent/consul/leader_connect_ca.go at the referenced failure and control-plane/subcommand/get-consul-client-ca/command_test.go around the retry loop. Compare the two proposed state-management options and confirm the test no longer leaves initialization unable to progress when configuration updates arrive early.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100