spring-cloud / spring-cloud/spring-cloud-config

Key authentication fails for default SSH repository when using multiple repositories with pattern matching

Open
#1,692 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Java
Stars
2k
Forks
1.3k
Avg merge
2d 59m
Merged PRs (30d)
16

Description

When using Pattern Matching and Multiple Repositories, the server doesn't seem to complete authentication with the SSH server, resulting in the repository not being found.

I have the following configuration:

spring:
  cloud:
    config:
      server:
        git:
          uri: git@github.com:evpaassen/scc-config-a.git
          ignore-local-ssh-settings: true
          private-key: |
            -----BEGIN RSA PRIVATE KEY-----
            MIIJJwIBAAKCAgEArxgcYggvomWVg/Q0yOd/ykdreoDyyPgeO3mnqoiCBOPYAqu7
            (...)
            KZHY2Ty2t4JlfkmilzugFR7ADaXvO1++xyEMx4t9RzClbJSukG60fSPRew==
            -----END RSA PRIVATE KEY-----
          repos:
            repo-b:
              pattern: b/*
              uri: git@github.com:evpaassen/scc-config-b.git
              ignore-local-ssh-settings: true
              private-key: |
                -----BEGIN RSA PRIVATE KEY-----
                MIIJKQIBAAKCAgEA1MyJU9FV3f0UqiAcax88VVZMkE6APbJ0XJParx3RtjpLlE1h
                (...)
                QRA+zPJMseOp2u9ch2eiODzpKyzuqmHpNwpivi8XZZIftF5no8sOIRh8po8a
                -----END RSA PRIVATE KEY-----

Repository scc-config-b connects fine, while the default one, scc-config-a does not. This exception shows up in the log:

2020-09-02 10:43:44.072  WARN 17812 --- [-192.168.99.119] .c.s.e.MultipleJGitEnvironmentRepository : Error occured cloning to base directory.

org.eclipse.jgit.api.errors.InvalidRemoteException: Invalid remote: origin
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:251) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:306) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:200) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.cloneToBasedir(JGitEnvironmentRepository.java:589) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.copyRepository(JGitEnvironmentRepository.java:564) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.createGitClient(JGitEnvironmentRepository.java:547) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.refresh(JGitEnvironmentRepository.java:268) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.JGitEnvironmentRepository.getLocations(JGitEnvironmentRepository.java:246) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository.getLocations(MultipleJGitEnvironmentRepository.java:146) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.AbstractScmEnvironmentRepository.findOne(AbstractScmEnvironmentRepository.java:57) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.MultipleJGitEnvironmentRepository.findOne(MultipleJGitEnvironmentRepository.java:187) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.environment.CompositeEnvironmentRepository.findOne(CompositeEnvironmentRepository.java:58) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.cloud.config.server.config.ConfigServerHealthIndicator.doHealthCheck(ConfigServerHealthIndicator.java:73) ~[spring-cloud-config-server-2.2.5.RELEASE.jar:2.2.5.RELEASE]
	at org.springframework.boot.actuate.health.AbstractHealthIndicator.health(AbstractHealthIndicator.java:82) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthIndicator.getHealth(HealthIndicator.java:37) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:71) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpoint.getHealth(HealthEndpoint.java:39) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:99) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpointSupport.getAggregateHealth(HealthEndpointSupport.java:110) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpointSupport.getContribution(HealthEndpointSupport.java:96) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:74) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpointSupport.getHealth(HealthEndpointSupport.java:61) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:65) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:55) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.2.8.RELEASE.jar:5.2.8.RELEASE]
	at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:77) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:121) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at org.springframework.boot.actuate.endpoint.jmx.EndpointMBean.invoke(EndpointMBean.java:96) ~[spring-boot-actuator-2.3.3.RELEASE.jar:2.3.3.RELEASE]
	at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:809) ~[na:na]
	at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) ~[na:na]
	at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1466) ~[na:na]
	at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307) ~[na:na]
	at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399) ~[na:na]
	at java.management.rmi/javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:827) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
	at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359) ~[na:na]
	at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200) ~[na:na]
	at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197) ~[na:na]
	at java.base/java.security.AccessController.doPrivileged(Native Method) ~[na:na]
	at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196) ~[na:na]
	at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562) ~[na:na]
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796) ~[na:na]
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677) ~[na:na]
	at java.base/java.security.AccessController.doPrivileged(Native Method) ~[na:na]
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: org.eclipse.jgit.errors.NoRemoteRepositoryException: git@github.com:evpaassen/scc-config-a.git: ERROR: Repository not found.

	at org.eclipse.jgit.transport.TransportGitSsh.cleanNotFound(TransportGitSsh.java:218) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:306) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:170) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:137) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:123) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1271) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:243) ~[org.eclipse.jgit-5.1.3.201810200350-r.jar:5.1.3.201810200350-r]
	... 55 common frames omitted

Observations

  • When using only one repository, it works fine.
  • When I set both config repositories to public on Github, it works fine, because they don't need key authentication.
  • When I change the configuration to the following, it works fine, even for the default repo (but that doesn't make sense to me, because SCC requires me to configure the uri for the default repository): This is actually not true. 
    spring:
  cloud:
    config:
      server:
        git:
          uri: git@github.com:evpaassen/scc-config-a.git
          ignore-local-ssh-settings: true
          private-key: |
            -----BEGIN RSA PRIVATE KEY-----
            MIIJJwIBAAKCAgEArxgcYggvomWVg/Q0yOd/ykdreoDyyPgeO3mnqoiCBOPYAqu7
            (...)
            KZHY2Ty2t4JlfkmilzugFR7ADaXvO1++xyEMx4t9RzClbJSukG60fSPRew==
            -----END RSA PRIVATE KEY-----
          repos:
            repo-b:
              pattern: b/*
              uri: git@github.com:evpaassen/scc-config-b.git
              ignore-local-ssh-settings: true
              private-key: |
                -----BEGIN RSA PRIVATE KEY-----
                MIIJKQIBAAKCAgEA1MyJU9FV3f0UqiAcax88VVZMkE6APbJ0XJParx3RtjpLlE1h
                (...)
                QRA+zPJMseOp2u9ch2eiODzpKyzuqmHpNwpivi8XZZIftF5no8sOIRh8po8a
                -----END RSA PRIVATE KEY-----
            repo-a:
              pattern: a/*
              uri: git@github.com:evpaassen/scc-config-a.git
              ignore-local-ssh-settings: true
              private-key: |
                -----BEGIN RSA PRIVATE KEY-----
                MIIJJwIBAAKCAgEArxgcYggvomWVg/Q0yOd/ykdreoDyyPgeO3mnqoiCBOPYAqu7
                (...)
                KZHY2Ty2t4JlfkmilzugFR7ADaXvO1++xyEMx4t9RzClbJSukG60fSPRew==
                -----END RSA PRIVATE KEY-----

Sample project
I have created a sample project which demonstrates the issue: https://github.com/evpaassen/scc-server. The application connects to to two private repositories, for which I included the deploy keys in the project. The issue should be easily reproducable by just running the ConfigServerApplication.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in JGitEnvironmentRepository.java at cloneToBasedir and compare the authentication setup used for the default repository with MultipleJGitEnvironmentRepository.java's repository selection. Reproduce the YAML configuration with separate private keys and verify that the default SSH repository authenticates and is found when pattern-matched repositories are also configured.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, java, spring, spring-boot
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.