hashicorp / hashicorp/consul

Service registered using `/v1/catalog/register` are removed during anti-entropy sync

Open
#7,513 2 comments 0 reactions 0 assignees View on GitHub
theme/operator-usability
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
2d 6h
Merged PRs (30d)
43

Description

Services registered using `/v1/catalog/register` get removed during the anty-entropy sync. As far as I can tell it does not happen when using the agent endpoint.

I noticed this will looking into https://github.com/terraform-providers/terraform-provider-consul/issues/187

#### Reproduction Steps

```
$ consul agent -dev >consul.log &
[1] 41646
$ curl -X PUT -d '{"Node":"Remis-MacBook-Pro.local", "Address":"127.0.0.1", "Service":{ "Service":"redis", "Port":6379}}' http://localhost:8500/v1/catalog/register

true
$ curl http://localhost:8500/v1/catalog/services
{
"consul": [],
"redis": []
}
$ curl http://localhost:8500/v1/catalog/services # one minute later
{
"consul": []
}
```

### Consul log

```
==> Starting Consul agent...
Version: 'v1.7.2'
Node ID: '38fac0ab-c53d-29cd-367d-164559a3c5d1'
Node name: 'Remis-MacBook-Pro.local'
Datacenter: 'dc1' (Segment: '')
Server: true (Bootstrap: false)
Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false

==> Log data will now stream in as it occurs:

2020-03-26T15:13:08.268+0100 [DEBUG] agent: Using random ID as node ID: id=38fac0ab-c53d-29cd-367d-164559a3c5d1
2020-03-26T15:13:08.269+0100 [WARN] agent: Node name will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes.: node_name=Remis-MacBook-Pro.local
2020-03-26T15:13:08.269+0100 [DEBUG] agent.tlsutil: Update: version=1
2020-03-26T15:13:08.270+0100 [DEBUG] agent.tlsutil: OutgoingRPCWrapper: version=1
2020-03-26T15:13:08.270+0100 [INFO] agent.server.raft: initial configuration: index=1 servers="[{Suffrage:Voter ID:38fac0ab-c53d-29cd-367d-164559a3c5d1 Address:127.0.0.1:8300}]"
2020-03-26T15:13:08.270+0100 [INFO] agent.server.raft: entering follower state: follower="Node at 127.0.0.1:8300 [Follower]" leader=
2020-03-26T15:13:08.271+0100 [INFO] agent.server.serf.wan: serf: EventMemberJoin: Remis-MacBook-Pro.local.dc1 127.0.0.1
2020-03-26T15:13:08.271+0100 [INFO] agent.server.serf.lan: serf: EventMemberJoin: Remis-MacBook-Pro.local 127.0.0.1
2020-03-26T15:13:08.271+0100 [INFO] agent.server: Adding LAN server: server="Remis-MacBook-Pro.local (Addr: tcp/127.0.0.1:8300) (DC: dc1)"
2020-03-26T15:13:08.271+0100 [INFO] agent.server: Handled event for server in area: event=member-join server=Remis-MacBook-Pro.local.dc1 area=wan
2020-03-26T15:13:08.271+0100 [INFO] agent: Started DNS server: address=127.0.0.1:8600 network=tcp
2020-03-26T15:13:08.271+0100 [INFO] agent: Started DNS server: address=127.0.0.1:8600 network=udp
2020-03-26T15:13:08.272+0100 [INFO] agent: Started HTTP server: address=127.0.0.1:8500 network=tcp
2020-03-26T15:13:08.272+0100 [INFO] agent: Started gRPC server: address=127.0.0.1:8502 network=tcp
2020-03-26T15:13:08.272+0100 [INFO] agent: started state syncer
==> Consul agent running!
2020-03-26T15:13:08.338+0100 [WARN] agent.server.raft: heartbeat timeout reached, starting election: last-leader=
2020-03-26T15:13:08.339+0100 [INFO] agent.server.raft: entering candidate state: node="Node at 127.0.0.1:8300 [Candidate]" term=2
2020-03-26T15:13:08.339+0100 [DEBUG] agent.server.raft: votes: needed=1
2020-03-26T15:13:08.339+0100 [DEBUG] agent.server.raft: vote granted: from=38fac0ab-c53d-29cd-367d-164559a3c5d1 term=2 tally=1
2020-03-26T15:13:08.339+0100 [INFO] agent.server.raft: election won: tally=1
2020-03-26T15:13:08.339+0100 [INFO] agent.server.raft: entering leader state: leader="Node at 127.0.0.1:8300 [Leader]"
2020-03-26T15:13:08.339+0100 [INFO] agent.server: cluster leadership acquired
2020-03-26T15:13:08.339+0100 [INFO] agent.server: New leader elected: payload=Remis-MacBook-Pro.local
Processing server acl mode for: Remis-MacBook-Pro.local - 0
2020-03-26T15:13:08.339+0100 [INFO] agent.server: Cannot upgrade to new ACLs: leaderMode=0 mode=0 found=true leader=127.0.0.1:8300
2020-03-26T15:13:08.340+0100 [DEBUG] connect.ca.consul: consul CA provider configured: id=07:80:c8:de:f6:41:86:29:8f:9c:b8:17:d6:48:c2:d5:c5:5c:7f:0c:03:f7:cf:97:5a:a7:c1:68:aa:23:ae:81 is_primary=true
2020-03-26T15:13:08.351+0100 [INFO] agent.server.connect: initialized primary datacenter CA with provider: provider=consul
2020-03-26T15:13:08.351+0100 [INFO] agent.leader: started routine: routine="CA root pruning"
2020-03-26T15:13:08.351+0100 [DEBUG] agent.server: Skipping self join check for node since the cluster is too small: node=Remis-MacBook-Pro.local
2020-03-26T15:13:08.351+0100 [INFO] agent.server: member joined, marking health alive: member=Remis-MacBook-Pro.local
2020-03-26T15:13:08.525+0100 [DEBUG] agent: Skipping remote check since it is managed automatically: check=serfHealth
2020-03-26T15:13:08.525+0100 [INFO] agent: Synced node info
2020-03-26T15:13:08.525+0100 [DEBUG] agent: Node info in sync
2020-03-26T15:13:08.724+0100 [DEBUG] agent: Skipping remote check since it is managed automatically: check=serfHealth
2020-03-26T15:13:08.724+0100 [DEBUG] agent: Node info in sync
2020-03-26T15:13:10.341+0100 [DEBUG] agent.tlsutil: OutgoingRPCWrapper: version=1
2020-03-26T15:13:39.142+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/catalog/register from=127.0.0.1:49649 latency=629.466µs
2020-03-26T15:14:08.347+0100 [DEBUG] agent.server: Skipping self join check for node since the cluster is too small: node=Remis-MacBook-Pro.local
2020-03-26T15:14:34.208+0100 [DEBUG] agent.http: Request finished: method=GET url=/v1/catalog/services from=127.0.0.1:49676 latency=85.537µs
2020-03-26T15:14:54.679+0100 [DEBUG] agent: Skipping remote check since it is managed automatically: check=serfHealth
2020-03-26T15:14:54.680+0100 [INFO] agent: Synced node info
2020-03-26T15:14:54.680+0100 [INFO] agent: Deregistered service: service=redis
2020-03-26T15:15:08.285+0100 [DEBUG] agent.server.router.manager: Rebalanced servers, new active server: number_of_servers=1 active_server="Remis-MacBook-Pro.local.dc1 (Addr: tcp/127.0.0.1:8300) (DC: dc1)"
2020-03-26T15:15:08.352+0100 [DEBUG] agent.server: Skipping self join check for node since the cluster is too small: node=Remis-MacBook-Pro.local
2020-03-26T15:15:27.856+0100 [DEBUG] agent.http: Request finished: method=GET url=/v1/catalog/services from=127.0.0.1:49694 latency=63.706µs
2020-03-26T15:15:42.514+0100 [INFO] agent: Caught: signal=interrupt
2020-03-26T15:15:42.514+0100 [INFO] agent: Graceful shutdown disabled. Exiting
2020-03-26T15:15:42.514+0100 [INFO] agent: Requesting shutdown
2020-03-26T15:15:42.514+0100 [INFO] agent.server: shutting down server
2020-03-26T15:15:42.514+0100 [DEBUG] agent.leader: stopping routine: routine="CA root pruning"
2020-03-26T15:15:42.514+0100 [WARN] agent.server.serf.lan: serf: Shutdown without a Leave
2020-03-26T15:15:42.514+0100 [DEBUG] agent.leader: stopped routine: routine="CA root pruning"
2020-03-26T15:15:42.514+0100 [WARN] agent.server.serf.wan: serf: Shutdown without a Leave
2020-03-26T15:15:42.514+0100 [INFO] agent.server.router.manager: shutting down
2020-03-26T15:15:42.514+0100 [INFO] agent: consul server down
2020-03-26T15:15:42.514+0100 [INFO] agent: shutdown complete
2020-03-26T15:15:42.514+0100 [INFO] agent: Stopping server: protocol=DNS address=127.0.0.1:8600 network=tcp
2020-03-26T15:15:42.514+0100 [INFO] agent: Stopping server: protocol=DNS address=127.0.0.1:8600 network=udp
2020-03-26T15:15:42.514+0100 [INFO] agent: Stopping server: protocol=HTTP address=127.0.0.1:8500 network=tcp
2020-03-26T15:15:42.515+0100 [INFO] agent: Waiting for endpoints to shut down
2020-03-26T15:15:42.515+0100 [INFO] agent: Endpoints down
2020-03-26T15:15:42.515+0100 [INFO] agent: Exit code: code=1
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.