hashicorp / hashicorp/nomad

Keep nomad client running even some of host volumes do not exist

Open
#10,137 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

theme/storage type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

Proposal

The nomad client can not start up when some of the volumes are invalidation.
Since there are other jobs that do not depend on these lost volumes, It is good to continue running the nomad client.

Mar 09 10:43:30 c1 nomad[2201]: ==> Starting Nomad agent...
Mar 09 10:43:30 c1 nomad[2201]: ==> Error starting agent: client setup failed: node setup failed: failed to validate volume acme, err: stat /datas/acme: no such device
Use-cases

We have some stateful jobs and some stateless jobs, when volume is not available, the stateful jobs can't run,
but the stateless jobs should not affect.

Attempted Solutions

I do not familiar with nomad source code, so the idea may not fit, just an idea:

  • do not check volume on startup
  • only check volume state on plan and run jobs.

thanks all.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing Nomad client startup through node setup and host-volume validation, then compare that path with volume checks during job planning and execution. The issue does not name files or tests; done would mean the client remains running when some volumes are unavailable while jobs requiring them are prevented from running.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.