hashicorp / hashicorp/nomad

Agent Configuration Linter

Open
#9,947 2 comments 5 reactions 0 assignees View on GitHub
stage/needs-discussion theme/api theme/cli theme/config theme/security type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

*The origin of this idea was security focused, but it could be extended to non-security-related configuration issues.* Given this origin and the [complexity of securely configuring Nomad](https://www.nomadproject.io/docs/internals/security#secure-configuration), this issue will focus on security issues for now.

## Prior Art

Of course @jrasell has not only beat me to thinking about this, but has already written up a proof of concept:

https://github.com/jrasell/nomad-sherlock

## Proposal

Create a new command and API for linting agent configurations. The following checks should be implemented:

### Checks

- ACLs are enabled
- mTLS is enabled
- raw_exec is disabled
- Task Driver’s are not allowed full host filesystem access
- Docker Driver is configured with privileged=false
- This means CSI cannot currently be run in a fully secured cluster (without Sentinel)

### Implementation

The good news is that the checks may all be determined by statically analyzing the agent's configuration.

The bad news is:

1. Our agent configuration defaulting, parsing, merging, and normalizing code is intermingled with agent startup and would need extensive refactoring.
2. Some checks are task driver plugin specific and rely on the client's plugin loading code to run.

#### Refactoring Agent Configuration

To address `#1` above agent configuration should be refactored until the following method can be defined:

```go
func (c *Config) Lint() ([]*Warnings, error) {
// implement checks above
}
```

`Lint` must only be called once all defaulting, parsing, environment variables, command line flags, merging, and normalizing has taken place.

#### Driver Checks

An initial implementation could parse and look for builtin task driver settings that are insecure.

Future work could entail refactoring plugin loading out of the client agent as well as adding an optional linting API to task driver plugins.

#### CLI

The CLI is intended to be used *without* a Nomad cluster running. This way it can be incorporated into Packer/Terraform pipelines.

The CLI would perform something like (names, flags, and wording TBD):

```
$ nomad lint-agent path/to/secure/configs; echo $?
0

$ nomad lint-agent -verbose path/to/secure/configs
- [x] ACLs are enabled
- [x] mTLS is enabled
- [x] raw_exec is disabled
- [x] Task Driver’s are not allowed full host filesystem access
- [x] Docker Driver is configured with privileged=false

$ nomad lint-agent path/to/INSECURE/configs; echo $?
Warning: mTLS is disabled.
Warning: Task Driver’s are allowed full host filesystem access.
1

$ nomad lint-agent -verbose path/to/INSECURE/configs
- [x] ACLs are enabled
- [ ] mTLS is enabled
- [x] raw_exec is disabled
- [ ] Task Driver’s are not allowed full host filesystem access
- [x] Docker Driver is configured with privileged=false
```

Note that in this example output the CLI has different output for "good" and "bad" configurations which are configuration dependent. This implies implementation details which must be considered when writing `Config.Lint`.

#### API

An API would provide a JSON equivalent to the verbose CLI listing.

## Future Work

The API form could check configurations against jobs and list "offenders" for any job requiring insecure configuration.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by mapping the agent configuration defaulting, parsing, environment-variable, flag, merging, and normalizing flow described in the issue, then identify where a Config.Lint method could run after those stages. Review the proposed security checks and the existing agent CLI and API structure; done means agreeing on the refactoring scope and producing a cluster-independent lint command and JSON API with the shown pass/fail behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, cli, devops, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.