GoogleContainerTools / GoogleContainerTools/skaffold
Build in cluster which is not GCR via Kaniko throws an error: "No matching credentials were found"
- Dominant language
- Go
- Stars
- 15.9k
- Forks
- 1.7k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 10
Description
### Expected behavior
It is clearly described how to build and run in a remote Kubernetes cluster. Afterwards:
1. Kaniko builds the image in kaniko namespace
2. Kaniko pushes the image
3. Skaffold deploys the image in the dedicated dev namespace that I specify via HELM (This doesn't work as Skaffold tries to pull the image which it isn't allowed to do)
4. Skaffold redeploys on change
This is an amazing feature in an amazing tool so I am just wondering why this is so confusingly documented?
### Actual behavior
While the push secret does work, the pull secret required by Kaniko is very unclear. I tried a lot of things on setting the secret, including setting the relevant keys pullSecret*** in the skaffold.yaml, mounting the secret into the Kaniko pod and so on.
Currently I get the following error messages:
INFO[0124] Pushing image to ***.azurecr.io/***:3021198-dirty
INFO[0129] Pushed image to 1 destinations
ERRO[0144] No matching credentials were found for "***.azurecr.io"
ERRO[0144] No matching credentials were found for "***.azurecr.io"
getting image: GET https://***.azurecr.io/oauth2/token?scope=repository%***%3Apull&service=***.azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.
The only thing that worked curiously was to `docker login ****.azurecr.io -p password -u userName` on the client machine which is running the skaffold dev command. I am wondering the following:
- Why does skaffold need two secrets to build an image?
- Why does a "docker login" on the client machine fix the error? The cluster is pulling the image after all
- What is the definitive type the pullSecret should be configured as?
- Does Kaniko need the pullSecret after all? Why not just give it to skaffold so that the image can be pulled after building and pushing?
### Information
- Skaffold version: Most current
- Operating system: OS X
- Installed via: Homebrew
- Contents of skaffold.yaml:
I am starting the following yaml via: skaffold dev --port-forward --kube-context k8-nextor-dev -p remote,dev1
```yaml
apiVersion: skaffold/v2beta26
kind: Config
build:
local:
push: false
useDockerCLI: true
useBuildkit: true
artifacts:
- image: ******.azurecr.io/******
context: .
docker:
dockerfile: Dockerfile
target: local
sync:
manual:
# Sync all folders relevant for application development
- src: '**/*.py'
dest: .
deploy:
kubeContext: docker-desktop
helm:
releases:
- name: ******-local-scheduler
chartPath: ../******-infrastructure/helm/******/localcharts/scheduler/
valuesFiles: ["../******-infrastructure/helm/******/values.yaml"]
namespace: ******-local
artifactOverrides:
image: ******.azurecr.io/scheduler
flags:
upgrade: ["--dependency-update"]
profiles:
- name: remote
deploy:
kubeContext: remote-cluster
patches:
- op: add
path: /deploy/helm/releases/0/valuesFiles/-
value: "../*******-infrastructure/helm/******/values-remote.yaml"
build:
cluster:
namespace: kaniko
# Here I tried all kinds of secrets...
dockerConfig:
secretName: kaniko-push-secret
artifacts:
- image: *****.azurecr.io/*****
context: .
kaniko:
dockerfile: Dockerfile
target: local
skipUnusedStages: true
useNewRun: true
sync:
manual:
- src: '**/*.py'
dest: .
- name: dev1
patches:
- op: replace
path: /deploy/helm/releases/0/namespace
value: remote-dev1-namespace
- op: add
path: /deploy/helm/releases/0/valuesFiles/-
value: "../******-infrastructure/helm/******/remote/dev1.yaml"
```
### Steps to reproduce the behavior
1. a clonable repository with the sample skaffold project
2. `skaffold `
3. ...
Contributor guide
Assessment
This issue has not been assessed yet.