carvel-dev / carvel-dev/vendir
vendir support for "dot" paths without conflict
- Dominant language
- Go
- Stars
- 399
- Forks
- 73
- Avg merge
- 2h 53m
- Merged PRs (30d)
- 2
Description
**Describe the problem/challenge you have**
vendir's requirement that paths do not overlap makes it difficult, or impossible, to create a directory structure that custom ytt modules can be found in if there are also environment specific yamls which need to be included.
consider the following.
* a _ytt_lib directory needs to be included from a git location
* some number of blobs of yaml (which i'll just call "modules" for now) need to be included
* some number of environment specific yaml one-off files need to be included.
The final directory structure that we're looking to create wants to look like this
```
├── config
├── _ytt_lib
│ └── ambassador-mappings
│ └── deployment.yaml
├── modules
│ ├── base
│ │ ├── defaults.yaml
│ │ ├── deployment.yaml
│ │ ├── namespace.yaml
│ │ ├── service-account.yaml
│ │ └── service.yaml
│ ├── cdm-subscription
│ │ └── deployment.yaml
│ └── ...
├── secret.enc.yaml
├── terraform.yaml
└── values.yaml
```
The above directory layout does not seem to be possible due to a constraint in vendir; `Expected to not manage overlapping paths: 'config/modules' and 'config'`
The vendir.yml file we're making using of looks roughly like the following.
```
---
apiVersion: "vendir.k14s.io/v1alpha1"
kind: "Config"
directories:
- path: "config"
contents:
- path: "."
git:
url: "https://gitlab.foo.bar/vss-cloud-accounts-service-infra.git"
ref: "v1.28.1"
includePaths:
- "k8s/envs/stg-01/secret.enc.yaml"
newRootPath: "k8s/envs/stg-01"
- path: "config/service"
contents:
- path: "modules"
git:
url: "https://gitlab.foo.bar/vss-cloud-accounts-service-infra.git"
ref: "v1.28.1"
includePaths:
- "k8s/modules/base/*"
- "k8s/modules/cdm-subscription/*"
- "k8s/modules/cloud-resources/*"
- "k8s/modules/cloud-account-credentials/*"
- "k8s/modules/csp/*"
- "k8s/modules/data-reconciler/*"
- "k8s/modules/flagger-canary/*"
- "k8s/modules/inventory-new-accounts/*"
- "k8s/modules/irsa-service-account-annotation/*"
- "k8s/modules/k8s-collector/*"
- "k8s/modules/kapp-config/*"
- "k8s/modules/legacy-changes/*"
- "k8s/modules/pod-disruption-budget/*"
- "k8s/modules/service-annotations/*"
- "k8s/modules/service-endpoints/*"
- "k8s/modules/wavefront/*"
newRootPath: "k8s/modules"
- path: "config/_ytt_lib"
contents:
- path: "ambassador-mappings"
git:
url: "https://gitlab.foo.bar/vss-cloud-accounts-service-infra.git"
ref: "v1.28.1"
includePaths:
- "k8s/modules/ambassador-mappings/*"
newRootPath: "k8s/modules/ambassador-mappings"
```
the first blob is the blob at issue because it wants to add content to ".", but "." conflicts with the overlay constraint.
**Describe the solution you'd like**
I'd like vendir to allow this use case similar to how `gilt` allows it. For example, the following `gilt` configuration produces the desired result.
```
---
- git: https://gitlab.foo.bar/vss-cloud-accounts-service-infra.git
version: v1.28.1
files:
- src: k8s/modules
dst: config/modules
post_commands:
- rm -rf ambassador-mappings
- src: k8s/modules/ambassador-mappings
dst: config/_ytt_lib/ambassador-mappings
- src: k8s/envs/stg-01/*
dst: config/
post_commands:
- rm -rf vendir.yml
```
but gilt itself has its own constraints in terms of functionality. constraints that vendir doesnt have.
**Anything else you would like to add:**
part of the issue here, as far as I understand things, has to do with the `ytt` requirement that the `_ytt_lib` directory must be a sibling of any yaml files that use content in that directory.
If this `ytt` constraint did not exist, then there would be a clear path forward in how to use vendir. We could create any number of sub-directories and then ytt could either be told where it's `_ytt_lib` is, or, `ytt` could walk up the tree until it found a suitable `_ytt_lib` to use.
---
Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.
Contributor guide
Research direction
Start by reproducing the overlapping-path error with the vendir.yml configuration in the issue, then trace vendir's path-overlap validation. Define and test the intended behavior for a content path of "." alongside nested paths; done means the configuration succeeds and produces the requested config/modules, config/_ytt_lib, and environment-specific files without conflicts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, go
- Domain
- cli, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100