spring-cloud / spring-cloud/spring-cloud-config
Support Additional spring.config.import Statements In EnvironmentRepositories
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2k
- Forks
- 1.3k
- Avg merge
- 2d 59m
- Merged PRs (30d)
- 16
Description
Describe the bug
We are using the spring cloud config server to load configuration into our spring boot app using git and hashicorp vault.
The configuration for the server is like this
spring:
profiles:
active: git,vault
cloud:
config:
server:
native:
git:
uri: https://our-git-uri
searchPaths:
- '{application}'
vault:
host: our-vault-host
scheme: https
port: 443
authentication: our-authentication
kv-version: 2
If we ask for the config for the application 'app' with the active profile 'prof' it will load the 'app-prof.yml' configuration from git as expected. If the 'app-prof.yml' configuration being loaded contains vault imports like the following
spring:
config:
import: >-
vault://arbitrary/secret/path1
vault://arbitrary/secret/path2
Then the Spring cloud config server loads the configuration successfully and then proceeds to import the properties from the vault paths specified in the configuration, which is exactly what we want.
** The single bug is this - I would expect the properties loaded from the vault paths to be included in the configuration returned to the application that requested the configuration, but they are not. **
I investigated and below are explained some of the reasons I believe the properties are not returned. I do not know what the desired functionality is of each part and so do not know if what I list below are bugs, however they do result in the bug presented above manifesting.
There seem to be two barriers to these properties being returned in the response to the configuration request, both of which contribute to the single bug.
The PassthruEnvironmentRepository only 'passes through' configuration in MapPropertySources but the vault properties are stored in EnumerablePropertySources. it is possible to change this behaviour with the following code change
in PassthruEnvironmentRepository::findOne change MapPropertySource to EnumerablePropertySource
if (!this.standardSources.contains(name) && source instanceof EnumerablePropertySource<?>) {
result.add(
new PropertySource(name, getMap((EnumerablePropertySource<?>) source, includeOrigin), source));
}
then PasstheuEnvironmentRepository::getMap can be simplified and still work the same
private Map<?, ?> getMap(EnumerablePropertySource<?> source, boolean includeOrigin) {
Map<Object, Object> map = new LinkedHashMap<>();
if (includeOrigin && source instanceof OriginLookup) {
Map<?, ?> input = (Map<?, ?>) source.getSource();
OriginLookup<String> originLookup = (OriginLookup<String>) source;
for (Object key : input.keySet()) {
Origin origin = originLookup.getOrigin(key.toString());
if (origin == null) {
map.put(key, source.getProperty(key.toString()));
continue;
}
String originDesc;
if (origin instanceof TextResourceOrigin) {
TextResourceOrigin tro = (TextResourceOrigin) origin;
originDesc = tro.getLocation().toString();
}
else {
originDesc = origin.toString();
}
Object value = source.getProperty(key.toString());
map.put(key, new PropertyValueDescriptor(value, originDesc));
}
}
else {
for (String key : source.getPropertyNames()) {
// Spring Boot wraps the property values in an "origin" detector, so we
// need
// to extract the string values
map.put(key, source.getProperty(key));
}
}
return map;
}
As MapPropertySource extends EnumerablePropertySource this is backwards compatible and does not change the normal functionality, but now the properties returned from the arbitrary vault path import are available in the property source map.
The second barrier I have found is that the arbitrary vault paths being returned are removed by the NativeEnvironmentRepository. Even though we do not have the native profile enabled the NativeEnvironmentRepository is used to filter out what is returned. the NativeEnvironmentRepository::matchesLocation method does not return true for the arbitrary vault paths and I cannot fathom a clean way to make this actually happen. This is where I got stuck.
I am very happy with the way that the cloud config server chooses which configuration files to load but it we have a requirement that each active profile may load up different properties from arbitrary vault paths that have nothing to do with the application or profile of the configuration file that they are specifies in. We do not even control those paths on the app side.
It is very frustrating that it so nearly works exactly as we want, but that the properties get filtered out at the end because they are not linked to the active profile in the same way.
We would like to use the cloud-config-server as a one-stop-shop for loading configuration but this is stopping us.
Can this be fixed?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with PassthruEnvironmentRepository::findOne and getMap to trace how imported EnumerablePropertySources are converted, then inspect NativeEnvironmentRepository::matchesLocation to understand why arbitrary vault paths are filtered. Done means configuration imported from multiple arbitrary vault paths is included in the response while existing configuration filtering remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, java, spring, spring-boot
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100