Support Node Pool within Task Group Stanza
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Support `node_pool` at the Task Group level in jobs
#### Summary
Allow jobs to specify `node_pool` at the **task group** level (in addition to the existing job-level setting) so that different groups within the same job can target different node pools.
---
## Background / Problem Statement
Today, `node_pool` can be defined at the **job** level, but placement is ultimately decided per **allocation**, and allocations map to **task groups**. This creates a practical limitation: a single job cannot schedule multiple groups into different node pools.
As a result, users must split what is logically one application/job into **multiple Nomad jobs** just to target different node pools. This becomes painful when groups need coordinated rollout via deployments, because Nomad does not provide a mechanism to coordinate deployment dependencies across separate jobs the way it can within a single job.
---
## Goals
* Enable specifying `node_pool` at the **group** level so a single job can contain multiple groups targeting different node pools.
* Preserve existing behavior for jobs that only specify `node_pool` at the job level.
* Make placement intent clearer by aligning configuration with the allocation/task-group boundary.
---
## Proposed Behavior
* Add `node_pool` (or equivalent) configuration at the **task group** stanza.
* If `group.node_pool` is set, it takes precedence for allocations created from that group.
* If `group.node_pool` is not set, fall back to `job.node_pool` (existing behavior).
* If neither is set, behavior remains unchanged (default scheduling rules).
---
## Acceptance Criteria
1. **Group-level targeting works**
* Given a job with two task groups `groupA` and `groupB`:
* `groupA` configured with `node_pool = "pool-a"`
* `groupB` configured with `node_pool = "pool-b"`
* Allocations for `groupA` are only placed on nodes in `pool-a`, and allocations for `groupB` are only placed on nodes in `pool-b`.
2. **Backwards compatibility**
* A job that specifies `node_pool` only at the job level behaves exactly as it does today.
* Existing job specs continue to validate and run without modification.
3. **Precedence rules**
* When both job-level and group-level `node_pool` are present, the group-level value is used for that group’s allocations.
* This precedence is documented and covered by tests.
4. **User feedback on invalid configuration**
* If a group references a node pool that does not exist (or is not available), the job fails validation or produces a clear, actionable scheduler/plan error (whichever is consistent with existing Nomad behavior for node pool validation).
5. **Documentation**
* Job specification docs are updated to describe group-level `node_pool`, including examples and precedence with job-level `node_pool`.
---
## Example Use Case
A single application job defines:
* `api` group that must run in a “services” node pool
* `worker` group that must run in a “batch” node pool
Today this requires two separate Nomad jobs, making coordinated changes harder. Group-level `node_pool` enables this to remain one cohesive job.
---
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the existing job-level node_pool handling, allocation placement logic, and related validation tests. Implement group-level precedence while preserving job-level fallback, then verify placement, invalid configuration feedback, backward compatibility, and updated job specification documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100