hashicorp / hashicorp/consul

Remove artificial 64 operations-per-transaction limit

Open
#15,955 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
2d 6h
Merged PRs (30d)
43

Description

Please search the existing issues for relevant feature requests, and use the reaction feature (https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to add upvotes to pre-existing requests.

#### Feature Description

Remove the limit of only 64 operations per transaction

#### Use Case(s)

## 1
We sometimes need to perform lots of operations against Consul's key/value store. For example one recent task required performing about 55k key updates. Performing these updates one by one was pegging the CPU, and would have taken approximately 12 hours to complete. Switching to using transactions (performing 64 key updates per transaction) dramatically sped this task up, and it completed in about 20 minutes. But even this was still pegging the CPU, resulting in lots of error messages about timeouts within the cluster and from other nodes.

While looking at the source code, it seems this limit of only being able to perform 64 operations per transaction is an artificial limit, and not due to any actual protocol limitations or anything. https://github.com/hashicorp/consul/blob/85b20d371396feecd310600b8c7d60aa1c532dbd/agent/txn_endpoint.go#L121-L126

Seeing as performing multiple updates per transaction resulted in a massive performance increase, it stands to reason that if we were able to perform more operations per transaction, the performance would increase even further, reducing the period in which the CPU is pegged and erroring. If we could perform the entire task in a single transaction, we may be able to avoid the issue entirely (just a theory). Therefore, since there is no technical reason this limit exists, it should be removed.

## 2
Due to the limitation of only 64 operations per transaction, we've engineered our solution to minimize potential issues by performing all inserts first, then updates, and then deletes. For the most part, this works. But it would be safer if we could perform the operations in a single transaction.

## 3
We have several hundred consul-template instances subscribing to key/value updates, and every time an update happens, it results in consul-templating rebuilding the output file. With 64 updates per transaction, this isn't too bad, but it does result in unnecessary reload operations (for the software which is consuming the templated files).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.