hashicorp / hashicorp/nomad

Allow Variable Interpolation in resources block

Open
#20,063 2 comments 0 reactions 0 assignees View on GitHub
stage/thinking theme/jobspec type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

In our cluster, we need to run a job on every node. Ideally, the resources used per node should vary, e.g., when the job is running on a production node, more resources should be reserved. When the job runs on a staging node, we need fewer resources.

At first, I thought I can solve this by using a lookup function and pass the $node.pool variable to it. But after some experiments, I realized that node attributes are interpretable in a resources block:
```
variable "nodepool-memory" {
type = map(number)
default = {
prod = 2048
stage = 512
test = 128
}
}

job "my-job" {
node_pool = "all"
datacenters = ["dc1"]
type = "system"

group "my-job-group" {
task "my-job-task" {
# ...

resources {
cpu = 256
memory = lookup(var.nodepool-memory, "${node.pool}", 64)
}
}
}
}
```

The only option I currently saw, is using a group per node pool and restrict the allocation with a constraint to be scheduled on a specific node pool:
````
job "my-job" {
node_pool = "all"
datacenters = ["dc1"]
type = "system"

group "my-job-group-prod-pool" {
task "my-job-task" {
# ...

constraint {
attribute = "${node.pool}"
operator = "="
value = "master"
}

resources {
cpu = 256
memory = 2048
}
}
}

group "my-job-group-stage-pool" {
task "my-job-task" {
# ...

constraint {
attribute = "${node.pool}"
operator = "="
value = "master"
}

resources {
cpu = 256
memory = 512
}
}
}
}
````

The downside of this approach is, that the job file gets quite big and involves a lot of code duplication. The whole task definition - except the resources block - gets duplicated for each group, making it harder to maintain in the long run.

Is there any change to allow/enable variable interpolation for a resource block? Or is there any better way of managing this without duplicating code as much now? Maybe I am missing something obvious here.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how Nomad parses job, group, task, and resources blocks, then determine where variable interpolation is evaluated relative to node attributes. The work is done when a resources block can vary values by node pool without duplicating the task definition, with behavior and validation covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.