hashicorp / hashicorp/nomad

document recommended resources for `operator snapshot agent`

Open
#27,902 0 comments 0 reactions 0 assignees View on GitHub
hcc/jira theme/docs theme/enterprise type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

Nomad Enterprise has the [`nomad operator snapshot agent`](https://developer.hashicorp.com/nomad/commands/operator/snapshot/agent) command, which you can use to periodically snapshot your cluster and ship backups to AWS S3 or whatever. We recommend deploying `nomad operator snapshot agent` as a Nomad job, but we don't have a recommended resource configuration that's been actually benchmarked.

CPU probably doesn't matter much because at worst that just slows things down, but memory is harder to handwave because of OOM. Even if we assume we're efficiently streaming, the RSS is going to include the paged-in parts of the Nomad binary. But this hasn't been specifically optimized the way we've done for logmon, etc. The bulk of the functionality is over in https://github.com/hashicorp/raft-snapshotagent (internal repo), which in turn pulls in Azure/AWS/GCP clients, which are all pretty large.

Let's benchmark this on a real cluster with a real external object storage, so that we can make accurate recommendations here.

Ref: https://github.com/hashicorp/nomad/pull/27890
Ref (internal): https://hashicorp.atlassian.net/browse/NMD-1426

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the `nomad operator snapshot agent` command documentation and the Nomad job deployment described in the issue. Run it on a real cluster with external object storage, then measure memory/RSS and CPU during snapshots. Done means documenting benchmark-backed resource recommendations, especially memory guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
cloud, documentation, infrastructure
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.