facebook / facebook/zstd

Question: Confirming v1.4.1 decoder compatibility with v1.5.7 encoder output

Open
#4,705 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
C
Stars
27.9k
Forks
2.6k
Avg merge
1d 3h
Merged PRs (30d)
8

Description

This is a compatibility confirmation question, not a bug — I'm using this template because it's
the only one available. No decoding failure has been observed.

Prior art in this repository:

- #3110 "ZSTD compatibility" — @Cyan4973 confirmed with "Yes" that v1.5.2 output can be decoded
by v1.4.5; @terrelln stated "Zstd will always be compatible with older versions."
- #3711 "Compatibility between compress/decompress APIs" — asked in the RocksDB context;
@Cyan4973 confirmed "Already deployed decoders will have no problem decoding new frames with
checksum enabled."

Given the above official guarantees, I would like a targeted confirmation for our specific
downgrade scenario before executing it on a multi-TB production deployment.

Scenario

We are planning to upgrade zstd from v1.4.1 to v1.5.7 in a distributed RocksDB-based KV storage
cluster (zstd is statically linked into each node's binary). The upgrade will be done as a
rolling deployment across dozens of nodes; during the rolling window, the cluster is in a
mixed-version state:

1. Upgrade mid-state (unavoidable, lasts hours to days per cluster): nodes still on v1.4.1 must
be able to decode SSTs / raft snapshots produced by peers running v1.5.7. A single Raft
replication group typically contains 3 replicas that may straddle the version boundary.
2. Rollback scenario (safety-net requirement): if we need to roll a node back from v1.5.7 to
v1.4.1, that node's decoder must still handle data that was written while it was on v1.5.7.

Both cases boil down to the same technical question: "Can a v1.4.1 decoder correctly decompress
every frame produced by a v1.5.7 encoder under our configuration?"

Question

Given the above encoder-side constraints, can a v1.4.1 decoder be guaranteed to correctly
decompress every frame produced by the v1.5.7 encoder, including dictionary-compressed blocks?

Reading #3110 and #3711, our understanding is "yes." We would appreciate a one-line confirmation
for the record, so we can attach it to our internal rollback decision document.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.