Support Global Context Variables
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 175
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 134
Description
# Describe the Feature
I tried to use the new `templated import` feature to fix some boilerplate on the billing components and it didn't produce best results. See the last section with detailed example. The actual try PR is here https://github.com/cloudposse/infra-live/pull/247/files (templating is reverted now)
Ideally this proposal should cover all use cases of current templated imports (replacing it) + my use case of nitpick-customizing large structures + adding a way to cross-reference component vars.
I don't propose to have this implementation and current templated imports implementation in atmos at the same time - we should pick one.
# Proposed Behavior
```
import:
- catalog/eks/cluster
- catalog/eks/karpenter
- catalog/eks/karpenter-provisioner
- catalog/eks/efs
...
template_vars:
region: us-east-2
region_short: ue2
tenant: plat
stage: dev
stack: "{{ tenant }}-{{ region_short }}"
description: "Component instance {{ name }} at {{ stack }}"
components:
terraform:
eks/reloader:
template_vars:
cmd_options: "--debug false"
my_var: "test"
vars:
enabled: true
name: reloader
repository: "https://stakater.github.io/stakater-charts"
chart: "{{ name }}"
chart_version: "v1.0.1"
create_namespace: true
kubernetes_namespace: "{{ name }}"
tags:
Team: sre
Service: "{{ name }}"
Region: "{{ region }}"
Tenant: "{{ Tenant }}"
Stack: "{{ stack }}"
Description: "{{ description }}"
values:
# https://github.com/stakater/Reloader/tree/master/deployments/kubernetes/chart/reloader
rbac:
enabled: true
serviceAccount:
create: true
name: "{{ name }}"
resources:
limits:
cpu: "20m"
memory: "128Mi"
requests:
cpu: "10m"
memory: "64Mi"
```
Final yaml vars rendering (adding two additional steps to atmos):
1. Deep merge yaml as usual
2. Gather template vars from all levels, including component vars
```template_vars = root_template_vars.deep_merge(component_template_vars).deep_merge(component..vars)```.
3. Render component vars
```component..vars = component_vars.render(template_vars)```. This render is recursive, it repeats untill are templates cross references are rendered.
4. Pass rendered vars to terraform as usual
# Use Case
Replacing / enhancing templated import functionality introduced here https://github.com/cloudposse/atmos/issues/307
# Additional Context
#### Before templates
Component:
```
components:
terraform:
account-settings:
budgets:
- name: Daily AWS Cost
budget_type: COST
# TODO atmos template and set per account (Currently we copy the whole list to override)
limit_amount: 10 # Around 300$ per month - account defaults
limit_unit: USD
time_unit: DAILY
notification:
# Daily alerts only support actual notification types
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 80
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
- name: Total AWS Cost
budget_type: COST
limit_amount: 300 # Around 300$ per month - account defaults
limit_unit: USD
time_unit: MONTHLY
notification:
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: FORECASTED
threshold_type: PERCENTAGE
threshold: 80
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: FORECASTED
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
```
Import:
```
import:
- catalog/account-settings
components:
terraform:
account-settings:
budgets:
- name: Daily AWS Cost
...
limit_amount: 17 # Around 500$ per month - account defaults
...
- name: Total AWS Cost
...
limit_amount: 100
...
```
^ Forces me to copy the whole list just to customize two values , because lists are not deep merged.
#### With templates (current implementation)
Component:
```
components:
terraform:
account-settings:
budgets:
- name: Daily AWS Cost
budget_type: COST
# TODO atmos template and set per account (Currently we copy the whole list to override)
limit_amount: {{ .daily_cost_usd | default 2 }}
limit_unit: USD
time_unit: DAILY
notification:
# Daily alerts only support actual notification types
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 80
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
- name: Total AWS Cost
budget_type: COST
limit_amount: {{ .monthly_cost_usd | default 60 }}
limit_unit: USD
time_unit: MONTHLY
notification:
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: FORECASTED
threshold_type: PERCENTAGE
threshold: 80
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: FORECASTED
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
```
Import:
```
import:
- path: "catalog/account-settings"
context:
daily_cost_usd: 26
monthly_cost_usd: 800
```
^ Much less boilerplate, but:
- actual stack variables are partially moved into the imports section
- if this is a nested import through mixin or something - all these variables have be passed on every level of those imports -> more boilerplate again
#### With ansible-style templates (suggested implementation)
Component:
```
components:
terraform:
account-settings:
budgets:
- name: Daily AWS Cost
budget_type: COST
# TODO atmos template and set per account (Currently we copy the whole list to override)
limit_amount: {{ .daily_cost_usd | default 2 }}
limit_unit: USD
time_unit: DAILY
notification:
# Daily alerts only support actual notification types
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 80
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
- name: Total AWS Cost
budget_type: COST
limit_amount: {{ .monthly_cost_usd | default 60 }}
limit_unit: USD
time_unit: MONTHLY
notification:
- comparison_operator: GREATER_THAN
notification_type: ACTUAL
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: FORECASTED
threshold_type: PERCENTAGE
threshold: 80
subscribers:
- slack
- comparison_operator: GREATER_THAN
notification_type: FORECASTED
threshold_type: PERCENTAGE
threshold: 100
subscribers:
- slack
```
Import:
```
import:
- catalog/account-settings
template_vars: {}
components:
terraform:
account-settings:
template_vars:
daily_cost_usd: 26
monthly_cost_usd: 800
```
^ Much less boilerplate
- vars are still defined at component level
- no need to bypass across imports since this is deepmerged first and then rendered once.
- can cross reference other vars of the component
Cons:
- this does not template component names, only vars, but I can do that using component inheritance we currently have
Contributor guide
Assessment
This issue has not been assessed yet.