carvel-dev / carvel-dev/ytt

Schema not applied correctly in library when using library.with_data_values().

Open
#563 5 comments 0 reactions 0 assignees View on GitHub
helping with an issue
Dominant language
Go
Stars
1.9k
Forks
167
PR merge metrics
No merged PRs in 30d

Description

**What steps did you take and what happened:**

Run:

```
#!/bin/sh

set -x

mkdir -p resources-1/_ytt_lib/mylib

cat > resources-1/_ytt_lib/mylib/00-schema.yaml << EOF
#@data/values-schema
---
apiVersion: ""
kind: ""
#@schema/type any=True
metadata:
name: ""
uid: ""
#@schema/type any=True
spec: null
#@schema/type any=True
status: null
EOF

cat > resources-1/_ytt_lib/mylib/99-output.yaml << EOF
#@ load("@ytt:data", "data")
--- #@ data.values
EOF

cat > resources-1/99-output.yaml << EOF
#@ load("@ytt:data", "data")
#@ load("@ytt:struct", "struct")
#@ load("@ytt:library", "library")
#@ load("@ytt:template", "template")

#@ mylib = library.get("mylib")
#@ mylib_with_values = mylib.with_data_values(struct.decode(data.values))

--- #@ template.replace(mylib_with_values.eval())
EOF

cat > values-1.yaml << EOF
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2021-10-13T05:54:35Z"
labels:
kubernetes.io/metadata.name: default
name: default
resourceVersion: "201"
uid: 591eeb77-5c81-49de-8698-28ac704d0467
spec:
finalizers:
- kubernetes
status:
phase: Active
EOF

ytt -f resources-1 --data-values-file values-1.yaml
```

This results in the error:

```
ytt: Error:
- library.eval: Evaluating library 'mylib': Overlaying data values (in following order: additional data values): Document on line ?: Map item (key 'metadata') on line ?: Map item (key 'creationTimestamp') on line ?: Expected number of matched nodes to be 1, but was 0
in
99-output.yaml:9 | --- #@ template.replace(mylib_with_values.eval())
```

With the files created from the test, if you were instead to run:

```
ytt -f resources-1/_ytt_lib/mylib --data-values-file values-1.yaml
```

In other words, directly process the files which are in the library, it works fine and produces:

```
apiVersion: v1
kind: Namespace
metadata:
name: default
uid: 591eeb77-5c81-49de-8698-28ac704d0467
creationTimestamp: "2021-10-13T05:54:35Z"
labels:
kubernetes.io/metadata.name: default
resourceVersion: "201"
spec:
finalizers:
- kubernetes
status:
phase: Active
```

Note that there is no `#@data/values` file in the library.

Now run a second test where the only difference is that we add a `#@data/values` file to the library, but where it is only marked as such and has no actual values set:

```
#!/bin/sh

set -x

mkdir -p resources-2/_ytt_lib/mylib

cat > resources-2/_ytt_lib/mylib/00-schema.yaml << EOF
#@data/values-schema
---
apiVersion: ""
kind: ""
#@schema/type any=True
metadata:
name: ""
uid: ""
#@schema/type any=True
spec: null
#@schema/type any=True
status: null
EOF

cat > resources-2/_ytt_lib/mylib/00-values.yaml << EOF
#@data/values
---
EOF

cat > resources-2/_ytt_lib/mylib/99-output.yaml << EOF
#@ load("@ytt:data", "data")
--- #@ data.values
EOF

cat > resources-2/99-output.yaml << EOF
#@ load("@ytt:data", "data")
#@ load("@ytt:struct", "struct")
#@ load("@ytt:library", "library")
#@ load("@ytt:template", "template")

#@ mylib = library.get("mylib")
#@ mylib_with_values = mylib.with_data_values(struct.decode(data.values))

--- #@ template.replace(mylib_with_values.eval())
EOF

cat > values-2.yaml << EOF
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2021-10-13T05:54:35Z"
labels:
kubernetes.io/metadata.name: default
name: default
resourceVersion: "201"
uid: 591eeb77-5c81-49de-8698-28ac704d0467
spec:
finalizers:
- kubernetes
status:
phase: Active
EOF

ytt -f resources-2 --data-values-file values-2.yaml
```

This time the test works and we get same output as when had processed the files in the library directly.

So in order for `library.with_data_values()` to work, you need to have a `#@data/values` file present.

Be warned though, that the data values file cannot actually have any values set in it, or you get an error again as seen with test:

```
#!/bin/sh

set -x

mkdir -p resources-3/_ytt_lib/mylib

cat > resources-3/_ytt_lib/mylib/00-schema.yaml << EOF
#@data/values-schema
---
apiVersion: ""
kind: ""
#@schema/type any=True
metadata:
name: ""
uid: ""
#@schema/type any=True
spec: null
#@schema/type any=True
status: null
EOF

cat > resources-3/_ytt_lib/mylib/00-values.yaml << EOF
#@data/values
---
metadata:
name: "xxx"
EOF

cat > resources-3/_ytt_lib/mylib/99-output.yaml << EOF
#@ load("@ytt:data", "data")
--- #@ data.values
EOF

cat > resources-3/99-output.yaml << EOF
#@ load("@ytt:data", "data")
#@ load("@ytt:struct", "struct")
#@ load("@ytt:library", "library")
#@ load("@ytt:template", "template")

#@ mylib = library.get("mylib")
#@ mylib_with_values = mylib.with_data_values(struct.decode(data.values))

--- #@ template.replace(mylib_with_values.eval())
EOF

cat > values-3.yaml << EOF
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2021-10-13T05:54:35Z"
labels:
kubernetes.io/metadata.name: default
name: default
resourceVersion: "201"
uid: 591eeb77-5c81-49de-8698-28ac704d0467
spec:
finalizers:
- kubernetes
status:
phase: Active
EOF

ytt -f resources-3 --data-values-file values-3.yaml
```

Which again generates the error:

```
ytt: Error:
- library.eval: Evaluating library 'mylib': Overlaying data values (in following order: _ytt_lib/mylib/00-values.yaml, additional data values): Document on line ?: Map item (key 'metadata') on line ?: Map item (key 'creationTimestamp') on line ?: Expected number of matched nodes to be 1, but was 0
in
99-output.yaml:9 | --- #@ template.replace(mylib_with_values.eval())
```

**What did you expect:**

Expect the following:

* It should not be required to have an empty `#@data/values` annotated file to get it to work.
* Where the `#@data/values` file sets values it should not error and the values passed to `library.with_data_values()` should me merged just like if `--data-values-file` was used on the command line and it was overriding a data values file included with a set of YAML files where a library wasn't involved.

**Environment:**

```
% ytt version
ytt version 0.38.0
```

---
Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.