hashicorp / hashicorp/consul

The service metadata is successfully updated using the endpoint /v1/txn, but after tens of seconds, the metadata is synchronized back to the old value.

Open
#12,581 3 comments 0 reactions 1 assignee Claimed by @blake View on GitHub
theme/service-metadata type/bug
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

#### Overview of the Issue

I set a metadata using the `consul` client call endpoint `/v1/txn`, and the call result returns OK without error. I find that the metadata is the new value I expect through the endpoint `/v1/health/service/{name}`.but after tens of seconds, the metadata is synchronized back to the old value.
when `enable_tag_override` is true, the tag of the service can avoid being synchronized to the old value, but other fields will still synchronize the old value.

#### Reproduction Steps

1. Create a cluster with 1 client nodes and 3 server nodes
2. execute golang code
``` go
// 1. register HelloService
if err := consul.Agent().ServiceRegister(&api.AgentServiceRegistration{
ID: "HelloService:localhost:8989",
Name: "HelloService",
Port: 8989,
Address: "localhost",
EnableTagOverride: false,
}); err != nil {
panic(err)
}
// 2. query HelloService
entries, _, err := consul.Health().Service("HelloService", "", false, &api.QueryOptions{})
if err != nil {
panic(err)
}
// 3. Execute transaction update metadata
ops := api.TxnOps{&api.TxnOp{
Service: &api.ServiceTxnOp{
Verb: api.ServiceCAS,
Node: entry.Node.Node,
Service: api.AgentService{
Kind: entry.Service.Kind,
ID: entry.Service.ID,
Service: entry.Service.Service,
Tags: entry.Service.Tags
Meta: map[string]string{"x": "y"},
Port: entry.Service.Port,
Address: entry.Service.Address,
TaggedAddresses: entry.Service.TaggedAddresses,
Weights: entry.Service.Weights,
EnableTagOverride: entry.Service.EnableTagOverride,
ModifyIndex: entry.Service.ModifyIndex,
CreateIndex: entry.Service.CreateIndex,
//ContentHash: entry.Service.ContentHash,
Proxy: entry.Service.Proxy,
Connect: entry.Service.Connect,
},
},
}}

ok, resp, m, err := consul.Txn().Txn(ops, nil)
```
3. View error
Wait for a synchronization cycle, about tens of seconds. View endpoint `/v1/health/service/HelloService` again, the metadata is synchronized back to the old value.

### Consul info for both Client and Server

Client info

```
agent:
check_monitors = 0
check_ttls = 0
checks = 2
services = 2
build:
prerelease =
revision =
version = 1.6.9
consul:
acl = disabled
known_servers = 3
server = false
runtime:
arch = amd64
cpu_count = 24
goroutines = 50
max_procs = 24
os = linux
version = go1.16.9
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 8
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 38
members = 4
query_queue = 0
query_time = 1
```

Server info

```
agent:
check_monitors = 0
check_ttls = 0
checks = 1
services = 1
build:
prerelease =
revision =
version = 1.6.9
consul:
acl = disabled
bootstrap = false
known_datacenters = 1
leader = true
leader_addr = 10.10.x.x:7897
server = true
raft:
applied_index = 38124
commit_index = 38124
fsm_pending = 0
last_contact = 0
last_log_index = 38124
last_log_term = 10
last_snapshot_index = 32774
last_snapshot_term = 7
latest_configuration = [{Suffrage:Voter ID:78da6485-c1c4-f2c9-afb4-0bb42e5e6bed Address:10.10.x.y:7897} {Suffrage:Voter ID:a7953f17-832a-191a-0e0b-ddeaca57e7dc Address:10.10.x.y+1:7897} {Suffrage:Voter ID:baf90ac9-1d5c-6e0e-7612-b28b0ec64f95 Address:10.10.x.y+2:7897}]
latest_configuration_index = 1
num_peers = 2
protocol_version = 3
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Leader
term = 10
runtime:
arch = amd64
cpu_count = 32
goroutines = 111
max_procs = 32
os = linux
version = go1.16.9
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 8
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 38
members = 4
query_queue = 0
query_time = 1
serf_wan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 4
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 184
members = 3
query_queue = 0
query_time = 1
```

### Operating system and Environment details

Linux liubei02 4.20.4-1.el7.elrepo.x86_64 #1 SMP Tue Jan 22 18:39:11 EST 2019 x86_64 x86_64 x86_64 GNU/Linux

### Log Fragments

#### Consul Server
``` log
2022/03/21 16:08:50 [DEBUG] memberlist: Initiating push/pull sync with: 192.168.a.b:8401
2022/03/21 16:08:57 [DEBUG] memberlist: Stream connection from=10.10.x.y:23371
2022/03/21 16:08:57 [DEBUG] memberlist: Stream connection from=10.10.c.d:14368
2022/03/21 16:08:59 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
2022/03/21 16:08:59 [DEBUG] agent: Node info in sync
2022/03/21 16:09:00 [DEBUG] http: Request GET /v1/health/service/HelloService?passing=1&tag=SERVER (2.883324ms) from=10.10.30.43:56037
2022/03/21 16:09:00 [DEBUG] http: Request PUT /v1/txn (11.018625ms) from=10.10.c.d:56037
2022/03/21 16:09:01 [DEBUG] http: Request GET /v1/health/service/HelloService (1.971278ms) from=10.10.c.d:56038
2022/03/21 16:09:18 [DEBUG] memberlist: Stream connection from=10.10.x.y:63136
2022/03/21 16:09:18 [DEBUG] memberlist: Stream connection from=10.10.x.z:13126
2022/03/21 16:09:20 [DEBUG] memberlist: Initiating push/pull sync with: 10.10.x.z:8401
2022/03/21 16:09:35 [DEBUG] memberlist: Initiating push/pull sync with: 10.10.x.y:8402
2022/03/21 16:09:50 [DEBUG] memberlist: Initiating push/pull sync with: 192.168.a.b:8401
2022/03/21 16:09:53 [DEBUG] http: Request GET /v1/health/service/HelloService (1.775929ms) from=10.10.c.d:56038
2022/03/21 16:09:56 [DEBUG] tlsutil: OutgoingRPCWrapper with version 1
2022/03/21 16:09:56 [DEBUG] manager: Rebalanced 3 servers, next active server is liubei02.dev (Addr: tcp/10.10.j.k:7897) (DC: dev)
2022/03/21 16:10:18 [DEBUG] memberlist: Stream connection from=10.10.x.y:63300
2022/03/21 16:10:20 [DEBUG] memberlist: Initiating push/pull sync with: 192.168.a.b:8401
```

#### Consul Client
``` log
2022/03/21 16:09:21 [DEBUG] memberlist: Initiating push/pull sync with: 10.10.x.x:8401
2022/03/21 16:09:27 [DEBUG] memberlist: Stream connection from=10.10.x.x:46850
2022/03/21 16:09:27 [DEBUG] memberlist: Stream connection from=10.10.x.x:51236
2022/03/21 16:09:29 [DEBUG] agent: Check "service:HelloService:192.168.x.x:58100" is passing
2022/03/21 16:09:34 [DEBUG] manager: Rebalanced 3 servers, next active server is guanyu02 (Addr: tcp/10.10.x.x:7897) (DC: dev)
2022/03/21 16:09:41 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
2022/03/21 16:09:41 [INFO] agent: Synced service "HelloService:192.168.x.x:58100"
2022/03/21 16:09:41 [DEBUG] agent: Check "service:HelloService:192.168.x.x:58100" in sync
2022/03/21 16:09:41 [DEBUG] agent: Node info in sync
2022/03/21 16:09:50 [DEBUG] memberlist: Stream connection from=10.10.x.x:46286
2022/03/21 16:09:51 [DEBUG] memberlist: Initiating push/pull sync with: 10.10.x.x:8401
2022/03/21 16:09:57 [DEBUG] memberlist: Stream connection from=10.10.x.x:51688
2022/03/21 16:09:59 [DEBUG] agent: Check "service:HelloService:192.168.x.x:58100" is passing
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.