Consul Connect for sharded tasks
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Proposal
Allow for connecting multiple instances of the same task (eg. when job.group.count > 1) to each other.
### Example usage
```hcl
job "rabbitmq" {
...
group "rabbitmq" {
count = 3
network {
mode = "bridge"
port "rabbit" {}
}
service {
name = "rabbitmq"
port = "rabbit"
connect {
sidecar_service {
proxy {
upstreams {
self = true
local_bind_port = "5672"
}
}
}
}
}
}
task "rabbitmq" {
template {
data = <
### Use-cases
Clustered tasks such as the mentioned rabbitmq or other message brokers, databases/search engines such as elasticsearch and other HA services that need to communicate with each other.
### Attempted Solutions
- I have unsuccessfully tried using host network (on one nomad node, so running with count > 1 fails due to port collision) to gain access to Consul instead (rabbitmq has consul plugin)
- I've attempted to ask if there's something I'm missing here: https://discuss.hashicorp.com/t/rabbitmq-with-consul-connect-using-nomad/28018
and here: https://gitter.im/hashicorp-nomad/Lobby?at=6111f5f07bc44d0a47136194 unsuccessfully
- I've attempted various stuff that I don't even remember anymore :(
I don't know if this is something that has to be done on Nomad side or Consul side.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by determining whether sharded self-upstreams belong in Nomad’s Consul Connect integration or Consul, then define the behavior for NOMAD_UPSTREAM_SELF and validate it against the RabbitMQ-style clustered-task example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100