hashicorp / hashicorp/consul

Deregistering sidecar does not deregister the destination service

Open
#9,798 2 comments 0 reactions 0 assignees View on GitHub
theme/connect theme/kubernetes
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
2d 6h
Merged PRs (30d)
43

Description

#### Overview of the Issue

Running consul in a k8s cluster. Sometimes the sidecar deregister fails to run when a pod gets deleted.
The sidecar gets removed after the deregister_critical_service_after timeout, but the original service remains.

#### Reproduction Steps

Steps to reproduce this issue, eg:

1. Create a cluster with n client nodes n and n server nodes
1. Register a service with a sidecar (remove preStop deregister logic to easily reproduce)
1. Delete pod

#### Proposal

Option 1: Remove the service when its sidecar is deregistered

Option 2: Allow for an alias_service check to the service sidecar

I tried the following (Proxy Alias Check), but it says the service sidecar does not exist on the node:
```hcl
services {
id = "${SERVICE_ID}"
name = "test-service"
address = "${POD_IP}"
port = 8080
meta = {
pod-name = "${POD_NAME}"
}
enable_tag_override = true

# Expected this to work
checks {
name = "Proxy Alias"
alias_service = "${PROXY_SERVICE_ID}"
}
}

services {
id = "${PROXY_SERVICE_ID}"
name = "test-service-sidecar"
kind = "connect-proxy"
address = "${POD_IP}"
port = 20000
tags = []
meta = {
pod-name = "${POD_NAME}"
}
enable_tag_override = true

proxy {
destination_service_name = "test-service"
destination_service_id = "${SERVICE_ID}"
local_service_address = "127.0.0.1"
local_service_port = 12345
${init_container.value.upstreams}
}

checks {
name = "Proxy Public Listener"
tcp = "${POD_IP}:20000"
interval = "10s"
# Set deregister_critical_service_after to be super low to reproduce
deregister_critical_service_after = "10s"
}

checks {
name = "Destination Alias"
alias_service = "${SERVICE_ID}"
}
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.