hashicorp / hashicorp/nomad

Nomad can not use consul ingress-gateways because tasks use protocol tcp

Open
#8,647 26 comments 17 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

hcc/jira stage/accepted theme/consul/connect theme/networking type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

Nomad 0.11.1
Consul 1.8.2

Consul Ingress-Gateways support `tcp` and `http` listeners. Http listeners are preferred because they allow for multiple services to listen on a single port and use Host header identification.

## Problem

Nomad jobs default to service type of `tcp`. There does not appear to be a documented way to change a nomad job to use `http` as the service type. As a result the user will get the following error when they attempt to create a listener for it.

https://www.nomadproject.io/docs/job-specification/service

```
Error writing config entry ingress-gateway/ingress-ngproxy: Unexpected response code: 500 (rpc error making call: service "count-dashboard" has protocol "tcp", which does not match defined listener protocol "http")
```

## Steps to reproduce

1. Submit the standard count-dash example

count-dash.job

```
job "countdash" {
datacenters = ["dc1"]
group "api" {
network {
mode = "bridge"
}

service {
name = "count-api"
port = "9001"

connect {
sidecar_service {}
}
}

task "web" {
driver = "docker"
config {
image = "hashicorpnomad/counter-api:v1"
}
}
}

group "dashboard" {
network {
mode ="bridge"
port "http" {
static = 9002
to = 9002
}
}

service {
name = "count-dashboard"
port = "9002"
# This is slightly modified from the stock count-dash examples
# By adding an 'http' health check, the hope was to force nomad to use 'http' over 'tcp'
check {
name = "count-dashboard-health"
type = "http"
protocol = "http"
path = "/health"
port = 9002
interval = "10s"
timeout = "5s"

}
connect {
sidecar_service {
proxy {
upstreams {
destination_name = "count-api"
local_bind_port = 8080
}
}
}
}
}

task "dashboard" {
driver = "docker"
env {
COUNTING_SERVICE_URL = "http://${NOMAD_UPSTREAM_ADDR_count_api}"
}
config {
image = "hashicorpnomad/counter-dashboard:v1"
}
}
}
}
```

2. Create an ingress controller and register it with consul config

```
consul config write ingress-service.hcl
```
```

Listeners = [
{
Port = 8080
Protocol = "http"
Services = [
{
Name = "count-dashboard",
Hosts = ["count.example.com"]
}
]
}
]
```

### Expected result

The service should be added to the ingress controller

### Actual result

Consul throws this warning

```
Error writing config entry ingress-gateway/ingress-service: Unexpected response code: 500 (rpc error making call: service "count-dashboard" has protocol "tcp", which does not match defined listener protocol "http")
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the count-dash job and ingress-gateway configuration described in the issue, then inspect the Nomad service job specification and the linked service documentation. Determine how an HTTP service protocol can be selected and verify that the count-dashboard service is accepted by the HTTP ingress listener; document the supported configuration and its expected result.

Written by the indexing model from the issue text.

Assessment

Domain
backend, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.