hashicorp / hashicorp/nomad

Support Node Pool within Task Group Stanza

Open
#27,490 2 comments 1 reaction 0 assignees View on GitHub
hcc/jira theme/node-pools type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

### Support `node_pool` at the Task Group level in jobs

#### Summary

Allow jobs to specify `node_pool` at the **task group** level (in addition to the existing job-level setting) so that different groups within the same job can target different node pools.

---

## Background / Problem Statement

Today, `node_pool` can be defined at the **job** level, but placement is ultimately decided per **allocation**, and allocations map to **task groups**. This creates a practical limitation: a single job cannot schedule multiple groups into different node pools.

As a result, users must split what is logically one application/job into **multiple Nomad jobs** just to target different node pools. This becomes painful when groups need coordinated rollout via deployments, because Nomad does not provide a mechanism to coordinate deployment dependencies across separate jobs the way it can within a single job.

---

## Goals

* Enable specifying `node_pool` at the **group** level so a single job can contain multiple groups targeting different node pools.
* Preserve existing behavior for jobs that only specify `node_pool` at the job level.
* Make placement intent clearer by aligning configuration with the allocation/task-group boundary.

---

## Proposed Behavior

* Add `node_pool` (or equivalent) configuration at the **task group** stanza.
* If `group.node_pool` is set, it takes precedence for allocations created from that group.
* If `group.node_pool` is not set, fall back to `job.node_pool` (existing behavior).
* If neither is set, behavior remains unchanged (default scheduling rules).

---

## Acceptance Criteria

1. **Group-level targeting works**

* Given a job with two task groups `groupA` and `groupB`:

* `groupA` configured with `node_pool = "pool-a"`
* `groupB` configured with `node_pool = "pool-b"`
* Allocations for `groupA` are only placed on nodes in `pool-a`, and allocations for `groupB` are only placed on nodes in `pool-b`.

2. **Backwards compatibility**

* A job that specifies `node_pool` only at the job level behaves exactly as it does today.
* Existing job specs continue to validate and run without modification.

3. **Precedence rules**

* When both job-level and group-level `node_pool` are present, the group-level value is used for that group’s allocations.
* This precedence is documented and covered by tests.

4. **User feedback on invalid configuration**

* If a group references a node pool that does not exist (or is not available), the job fails validation or produces a clear, actionable scheduler/plan error (whichever is consistent with existing Nomad behavior for node pool validation).

5. **Documentation**

* Job specification docs are updated to describe group-level `node_pool`, including examples and precedence with job-level `node_pool`.

---

## Example Use Case

A single application job defines:

* `api` group that must run in a “services” node pool
* `worker` group that must run in a “batch” node pool

Today this requires two separate Nomad jobs, making coordinated changes harder. Group-level `node_pool` enables this to remain one cohesive job.

---

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the existing job-level node_pool handling, allocation placement logic, and related validation tests. Implement group-level precedence while preserving job-level fallback, then verify placement, invalid configuration feedback, backward compatibility, and updated job specification documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.