Nomad connect functionality not working with SELinux enabled
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Nomad version
Nomad v0.10.4 (f750636ca68e17dcd2445c1ab9c5a34f9ac69345)
### Operating system and Environment details
Fedora 31, with 18.09.8
### Issue
The envoy health check in consul stays red and `var/log/audit/audit.log` contains denials:
```
type=AVC msg=audit(1583672022.178:2020): avc: denied { write } for pid=70868 comm="envoy" name="consul_grpc.sock" dev="tmpfs" ino=676989 scontext=system_u:system_r:container_t:s0:c121,c146 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=sock_file permissive=0
```
After `sentenforce 0` the health check turns green.
### Reproduction steps
Run `nomad agent -dev` and `consul agent -dev` and deploy the job file from below
### Job file (if appropriate)
```
job "example" {
datacenters = ["dc1"]
type = "service"
update { max_parallel = 1 }
group "http1" {
network {
mode = "bridge"
port "http" { to=80 }
}
service {
port = "http"
name = "http1"
connect {
sidecar_service {}
}
}
task "http1" {
driver = "docker"
config { image = "nginx" }
}
}
group "http2" {
network {
mode = "bridge"
port "http" { to=80 }
}
service {
port = "http"
name = "http2"
connect {
sidecar_service {
proxy {
upstreams {
destination_name = "http1"
local_bind_port = 8080
}
}
}
}
}
task "http2" {
driver = "docker"
config { image = "nginx" }
}
}
}
```
Consul logs have:
```
2020-03-08T14:02:56.962+0100 [WARN] agent: Check socket connection failed: check=service:_nomad-task-6d05e4c5-b5d8-2941-c6a4-dc9bb1e675c6-group-http2-http2-http-sidecar-proxy:1 error="dial tcp 127.0.0.1:30124: connect: connection refused"
2020-03-08T14:02:56.963+0100 [WARN] agent: Check is now critical: check=service:_nomad-task-6d05e4c5-b5d8-2941-c6a4-dc9bb1e675c6-group-http2-http2-http-sidecar-proxy:1
2020-03-08T14:03:00.781+0100 [WARN] agent: Check socket connection failed: check=service:_nomad-task-a6ae689e-6b3c-206d-5b58-0562248a595c-group-http1-http1-http-sidecar-proxy:1 error="dial tcp 127.0.0.1:20423: connect: connection refused"
2020-03-08T14:03:00.781+0100 [WARN] agent: Check is now critical: check=service:_nomad-task-a6ae689e-6b3c-206d-5b58-0562248a595c-group-http1-http1-http-sidecar-proxy:1
```
The other logs do not contain anything interesting sadly.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the job with `nomad agent -dev` and `consul agent -dev` on Fedora with SELinux enforcing, then inspect `var/log/audit/audit.log` and the Consul health-check logs. Done means the Envoy sidecar health checks become green without disabling SELinux or producing the reported socket denial.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- devops, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100