ACL binding rules for Workload identites is not working as expected.
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 39
Description
ACL binding rules for Workload identites is not working as expected.
---
In our infrastructure, we are using an open-source version of Nomad and Consul.
We have enforced a restriction in the naming convention of the service, i.e., it should be in the format -. This has been done so that we do not end up having the same identical service name belonging to different jobs running in different namespaces.
restriction is enforced by creating a policy for teams in consul that only allows write access to services having the prefix "namespace-"".
```
service_prefix "platforms-" {
policy = "write"
}
```
We would like to enable WI in Nomad and Consul, and the same restriction needs to be enforced on the service name naming convention. because if we don't put the restriction, different teams can have the same service name running in different namespaces in Nomad, which would cause issues.
We tried adding expressions in the acl binding rule selector field. However, it's not working as expected.
#### Reproduction Steps
Please find the authmethod and binding rule configuration
**Auth method:**
```
Type: jwt
Description: JWT auth method for Nomad services and workloads
Config:
{
"BoundAudiences": [
"consul.test.io"
],
"ClaimMappings": {
"nomad_job_id": "nomad_job_id",
"nomad_namespace": "nomad_namespace",
"nomad_service": "nomad_service",
"nomad_task": "nomad_task"
},
"JWKSURL": "http://127.0.0.1:4646/.well-known/jwks.json",
"JWTSupportedAlgs": [
"RS256"
]
}
```
**Binding rule:**
```
8eba3cd0-12ca-8c60-e69f-7cb08b48c814:
AuthMethod: nomad-workloads
Description: Binding rule for Nomad tasks
BindType: role
BindName: ${value.nomad_namespace}-team
Selector: "nomad_service" not in value
f5a6e1c4-aeac-0d08-ed95-9c7dee49528d:
AuthMethod: nomad-workloads
Description: Binding rule for services registered from Nomad
BindType: service
BindName: ${value.nomad_service}
Selector: "nomad_service" in value and value.nomad_service contains value.nomad_namespace
```
**Nomad config file**
```
bind_addr = "127.0.0.1"
advertise {
http = "127.0.0.1"
rpc = "127.0.0.1"
serf = "127.0.0.1"
}
ports {
http = 4646
rpc = 4647
serf = 4648
}
datacenter = "local"
region = "london"
disable_update_check = true
ui {
enabled = true
label {
text = "Environment: TEST DataCenter: gb2"
}
}
plugin "docker" {
config {
endpoint = "unix:///var/run/docker.sock"
}
}
log_level = "INFO"
log_rotate_duration = "24h"
log_rotate_max_files = 7
log_json = true
acl {
enabled = true
}
client {
enabled = true
servers = [
"127.0.0.1:4647"
]
node_class = "default"
node_pool = "platforms"
}
server {
enabled = true
bootstrap_expect = 1
encrypt =
server_join {
retry_join = [
"127.0.0.1:4648"
]
}
}
consul {
address = "http://127.0.0.1:8500"
token =
service_identity {
aud = ["consul.test.io"]
ttl = "1h"
}
task_identity {
aud = ["consul.test.io"]
ttl = "1h"
}
}
```
**Nomad Job file**
```
job "nginx" {
namespace = "platforms"
node_pool = "platforms"
group "nginx-group" {
count = 1
network {
port "http" {
to = "80"
}
}
task "nginx-task" {
driver = "docker"
config {
image = "nginx:latest"
ports = ["http"]
}
service {
name = "platforms-nginx-service"
port = "http"
provider = "consul"
check {
type = "http"
port = "http"
path = "/"
interval = "2s"
timeout = "2s"
}
}
}
}
}
```
When trying to run above Job, it fails with an error:
> Setup Failure: failed to setup alloc: pre-run hook "consul" failed: 1 error occurred: * failed to derive Consul token for service platforms-nginx-service: Unexpected response code: 403 (Permission denied)
Please note that if I use the below selector in acl binding where namespace name is hardcoded, it works as expected.
`Selector: "nomad_service" in value and value.nomad_service contains "platforms"`
However, if we replace `platforms` with `value.nomad_namespace`, it fails.
-->
**Consul and nomad version info**
consul : v1.19.2
noamd : v1.9.3
Contributor guide
Research direction
Start with Consul ACL binding rule selector evaluation, using the supplied auth method, binding rules, Nomad configuration, and Nomad job as the reproduction. Compare the dynamic value.nomad_namespace expression with the hardcoded platforms selector and verify that the workload can derive a token while the service-name prefix restriction remains enforced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- authorization
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100