GoogleContainerTools / GoogleContainerTools/skaffold

Issue in post-renderer when using helm charts. Error: Kubernetes YAML: yaml: unmarshal errors

Open
#8,020 11 comments 0 reactions 0 assignees View on GitHub
deploy/helm kind/bug priority/p2
Dominant language
Go
Stars
15.9k
Forks
1.7k
Avg merge
3d 9h
Merged PRs (30d)
10

Description

I am trying to introduce skaffold to our ethereal engine stack (https://github.com/XRFoundation/XREngine). I am leveraging local deployment of minikube with helm charts. We already have our helm charts located at: https://github.com/XRFoundation/XREngine/tree/dev/packages/ops/xrengine.

I am facing issue as mentioned below. Though as I dig down into the issue it seems like the following command executed by skaffold introduces the issue:
`helm --kube-context minikube upgrade local --reuse-values --debug --version 4.3.0 packages/ops/xrengine --post-renderer /usr/local/bin/skaffold -f packages/ops/values/xrengine.values.yaml`

Though when I run this command in my terminal without `--post-renderer /usr/local/bin/skaffold` then it works fine.
`helm --kube-context minikube upgrade local --reuse-values --debug --version 4.3.0 packages/ops/xrengine -f packages/ops/values/xrengine.values.yaml`

So, I assume there is something in skaffold's post-renderer that is causing the issue.

### Expected behavior

It should upgrade the helm deployment successfully

### Actual behavior

It fails to upgrade the helm deployment on local minikube with error:

```
hanzlamateen@hanzla-PC:~/xrengine$ skaffold dev
Listing files to watch...
- root-builder
- xrengine
Generating tags...
- root-builder -> root-builder:v1.0.0-rc5-32-gdb10818f7b-dirty
- xrengine -> xrengine:v1.0.0-rc5-32-gdb10818f7b-dirty
Checking cache...
- root-builder: Found Locally
- xrengine: Found Locally
Tags used in deployment:
- root-builder -> root-builder:6a62c513506f6ba4ec9fe638fbf2f882dcea2ef9817d5b0444299ac85f18f05a
- xrengine -> xrengine:0d10bb485b1b73093f7a14ade70ff9d4de967f3b43f7b5c256b321aebb4d748a
Starting deploy...
upgrade.go:142: [debug] preparing upgrade for local
upgrade.go:507: [debug] reusing the old release's values
Error: UPGRADE FAILED: error while running post render on files: error while running command /usr/local/bin/skaffold. error output:
reading Kubernetes YAML: yaml: unmarshal errors:
line 62: mapping key "APP_HOST" already defined at line 14
line 63: mapping key "APP_URL" already defined at line 15
line 64: mapping key "AUTH_SECRET" already defined at line 16
line 65: mapping key "CERT" already defined at line 21
line 66: mapping key "INSTANCESERVER_DOMAIN" already defined at line 34
line 69: mapping key "KEY" already defined at line 35
line 70: mapping key "LOCAL" already defined at line 36
line 71: mapping key "LOCAL_STORAGE_PROVIDER" already defined at line 37
line 72: mapping key "LOCAL_STORAGE_PROVIDER_PORT" already defined at line 38
line 80: mapping key "STORAGE_AWS_ACCESS_KEY_ID" already defined at line 52
line 81: mapping key "STORAGE_AWS_ACCESS_KEY_SECRET" already defined at line 53
line 82: mapping key "STORAGE_CLOUDFRONT_DOMAIN" already defined at line 54
line 83: mapping key "STORAGE_PROVIDER" already defined at line 55
line 84: mapping key "STORAGE_S3_REGION" already defined at line 60
line 85: mapping key "STORAGE_S3_STATIC_RESOURCE_BUCKET" already defined at line 61
: exit status 1
```

### Information

- Skaffold version: v2.0.0
- Operating system: Ubuntu 22.04
- Installed via: skaffold.dev
- Contents of skaffold.yaml:

```yaml
apiVersion: skaffold/v3
kind: Config
metadata:
name: xrengine
build:
artifacts:
- image: root-builder
context: .
docker:
dockerfile: dockerfiles/package-root/Dockerfile-root
- image: xrengine
context: .
docker:
dockerfile: Dockerfile
buildArgs:
NODE_ENV: 'development'
MYSQL_HOST: 'host.minikube.internal'
MYSQL_PORT: '3304'
MYSQL_PASSWORD: 'password'
MYSQL_USER: 'server'
MYSQL_DATABASE: 'xrengine'
VITE_CLIENT_HOST: 'local.theoverlay.io'
VITE_SERVER_HOST: 'api-local.theoverlay.io'
VITE_FILE_SERVER: 'https://localhost:8642'
VITE_MEDIATOR_SERVER: 'https://authn.io'
VITE_INSTANCESERVER_HOST: 'instanceserver-local.theoverlay.io'
VITE_READY_PLAYER_ME_URL: 'https://xre.readyplayer.me'
VITE_DISABLE_LOG: 'false'
VITE_8TH_WALL: 'null'
VITE_LOGIN_WITH_WALLET: 'false'
deploy:
helm:
flags:
upgrade:
- --reuse-values
- --debug
releases:
- name: local
chartPath: packages/ops/xrengine
valuesFiles:
- packages/ops/values/xrengine.values.yaml
version: 4.3.0
```

### Steps to reproduce the behavior

1. Clone https://github.com/XRFoundation/XREngine
2. Run `npm install`
3. Run `npm run dev-reinit`
4. Run `bash scripts/build_minikube.sh`
5. Place above mentioned `skaffold.yaml` file in root of repo
6. Copy 'packages/ops/configs/local.template.values.yaml' to 'packages/ops/values/xrengine.values.yaml'
7. Run `skaffold dev`

Error Logs (skaffold dev -vdebug):
```
DEBU[0026] Running command: [helm --kube-context minikube upgrade local --reuse-values --debug --version 4.3.0 packages/ops/xrengine --post-renderer /usr/local/bin/skaffold -f packages/ops/values/xrengine.values.yaml] subtask=0 task=Deploy
upgrade.go:142: [debug] preparing upgrade for local
upgrade.go:507: [debug] reusing the old release's values
Error: UPGRADE FAILED: error while running post render on files: error while running command /usr/local/bin/skaffold. error output:
reading Kubernetes YAML: yaml: unmarshal errors:
line 62: mapping key "APP_HOST" already defined at line 14
line 63: mapping key "APP_URL" already defined at line 15
line 64: mapping key "AUTH_SECRET" already defined at line 16
line 65: mapping key "CERT" already defined at line 21
line 66: mapping key "INSTANCESERVER_DOMAIN" already defined at line 34
line 69: mapping key "KEY" already defined at line 35
line 70: mapping key "LOCAL" already defined at line 36
line 71: mapping key "LOCAL_STORAGE_PROVIDER" already defined at line 37
line 72: mapping key "LOCAL_STORAGE_PROVIDER_PORT" already defined at line 38
line 80: mapping key "STORAGE_AWS_ACCESS_KEY_ID" already defined at line 52
line 81: mapping key "STORAGE_AWS_ACCESS_KEY_SECRET" already defined at line 53
line 82: mapping key "STORAGE_CLOUDFRONT_DOMAIN" already defined at line 54
line 83: mapping key "STORAGE_PROVIDER" already defined at line 55
line 84: mapping key "STORAGE_S3_REGION" already defined at line 60
line 85: mapping key "STORAGE_S3_STATIC_RESOURCE_BUCKET" already defined at line 61
: exit status 1
helm.go:84: [debug] exit status 1
error while running command /usr/local/bin/skaffold. error output:
reading Kubernetes YAML: yaml: unmarshal errors:
line 62: mapping key "APP_HOST" already defined at line 14
line 63: mapping key "APP_URL" already defined at line 15
line 64: mapping key "AUTH_SECRET" already defined at line 16
line 65: mapping key "CERT" already defined at line 21
line 66: mapping key "INSTANCESERVER_DOMAIN" already defined at line 34
line 69: mapping key "KEY" already defined at line 35
line 70: mapping key "LOCAL" already defined at line 36
line 71: mapping key "LOCAL_STORAGE_PROVIDER" already defined at line 37
line 72: mapping key "LOCAL_STORAGE_PROVIDER_PORT" already defined at line 38
line 80: mapping key "STORAGE_AWS_ACCESS_KEY_ID" already defined at line 52
line 81: mapping key "STORAGE_AWS_ACCESS_KEY_SECRET" already defined at line 53
line 82: mapping key "STORAGE_CLOUDFRONT_DOMAIN" already defined at line 54
line 83: mapping key "STORAGE_PROVIDER" already defined at line 55
line 84: mapping key "STORAGE_S3_REGION" already defined at line 60
line 85: mapping key "STORAGE_S3_STATIC_RESOURCE_BUCKET" already defined at line 61

helm.sh/helm/v3/pkg/postrender.(*execRender).Run
helm.sh/helm/v3/pkg/postrender/exec.go:63
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
helm.sh/helm/v3/pkg/action/action.go:219
helm.sh/helm/v3/pkg/action.(*Upgrade).prepareUpgrade
helm.sh/helm/v3/pkg/action/upgrade.go:234
helm.sh/helm/v3/pkg/action.(*Upgrade).RunWithContext
helm.sh/helm/v3/pkg/action/upgrade.go:143
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:197
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.3.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.3.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.3.0/command.go:902
main.main
helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
runtime/proc.go:255
runtime.goexit
runtime/asm_amd64.s:1581
error while running post render on files
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
helm.sh/helm/v3/pkg/action/action.go:221
helm.sh/helm/v3/pkg/action.(*Upgrade).prepareUpgrade
helm.sh/helm/v3/pkg/action/upgrade.go:234
helm.sh/helm/v3/pkg/action.(*Upgrade).RunWithContext
helm.sh/helm/v3/pkg/action/upgrade.go:143
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:197
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.3.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.3.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.3.0/command.go:902
main.main
helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
runtime/proc.go:255
runtime.goexit
runtime/asm_amd64.s:1581
UPGRADE FAILED
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:199
github.com/spf13/cobra.(*Command).execute
github.com/spf13/cobra@v1.3.0/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/cobra@v1.3.0/command.go:974
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/cobra@v1.3.0/command.go:902
main.main
helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
runtime/proc.go:255
runtime.goexit
runtime/asm_amd64.s:1581
DEBU[0027] Running command: [tput colors] subtask=-1 task=DevLoop
DEBU[0027] Command output: [256
] subtask=-1 task=DevLoop
deploying "local": install: exit status 1
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.