[RFC] Model priviliged execution environments in the task-driver API
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
Note: This issue mostly contains initial background thoughts to prompt discussion and is not yet a well defined proposal.
### Background
Currently, Nomads permission model around _runtime_ permissions of a job exist only within the implementation of a driver. This means that we do not include them in our ACL system or take whether the features are enabled on a particular client into account in the scheduler.
This is fine if a Nomad cluster is a uniform fleet, but that is rarely the case in larger clusters, and currently requires users to add additional metadata to privileged clients, and constraints to jobs that require them. It also then allows _anyone_ with access to the `submit-job` permission in any namespace to get privileged access to those hosts.
As part of #5378 however, access to privileged containers will become more normal as CSI Plugins require privileged containers in order to be able to create new mounts in the system for publishing volumes. Although we operate on a _trusted operator_ security model, there are many valid cases where CSI plugins may want to be deployed, without granting trivial docker privilege escalation to all.
### Proposal
I'm proposing introducing a Nomad-level API for modelling process isolation levels. This change would introduce a `privileged` option at the `task` level in a Nomad Job, that would signal to Nomad that the job my only be placed on nodes where that driver exposes the `Privileged` execution capability. It would also allow the introduction of a `privileged-execution` capability to the ACL system.
#### Task Configuration
```hcl
task "foo" {
driver = "docker"
privileged = true
}
```
#### Driver API
This is currently mostly undefined, but it would mostly involve updating the `DriverCapabilities` to introduce a new field that plugins may opt in to, and introducing a privileged option to TaskConfig.
#### Opting nodes into privileged execution
Currently, Nomad requires you to configure drivers with support for privileged execution modes in a client configuration. After this change, you'll still be required to enable support on an individual client, but by default will require using the new configuration for privileged execution modes.
To allow for backwards compatibility and a cleaner upgrade path, we will also offer an option in driver configuration to retain the existing behavior for using privileged execution environments.
##### Docker
###### Example Config
```hcl
plugin "docker" {
config {
allow_privileged = true // existing field
legacy_privileged_behavior = true // allow the existing behavior to be used, default: false
}
}
```
##### Raw Exec
The Raw Exec driver will begin exposing an `Unconstrained` isolation capability when `legacy_privileged_behavior` is `false` which will require that a user has access to `privileged` execution modes.
```hcl
plugin "docker" {
config {
enabled = true // existing field
legacy_privileged_behavior = true // allow the existing behavior to be used, default: false
}
}
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the task-driver API definitions around DriverCapabilities and TaskConfig, then trace how the Docker and Raw Exec drivers expose privileged execution and how ACL permissions are modeled. Before implementation, turn the RFC into a decided proposal covering scheduler placement, client configuration, backwards compatibility, and acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- api, backend, infrastructure, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100