facebook / facebook/zstd

Question: Confirming v1.5.5 ↔ v1.5.7 bidirectional compatibility for ZSTD_compress_usingDict / ZSTD_decompress_usingDict with raw-content dictionaries

Abierto
#4,706 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
27.9k
Forks
2.6k
Merge medio
1 d 3 h
PR fusionados (30 d)
8

Descripción

This is a compatibility confirmation question, not a bug — using the bug template because it is the only one available. No decoding failure has been observed.

Prior art in this repository:
- #3110 "ZSTD compatibility" — @Cyan4973 confirmed frame-format stability across versions.
- #3711 "Compatibility between compress/decompress APIs" — confirmed decoders are compatible with frames produced by newer encoders.
- #3802 "Compressing and decompressing with dictionaries, between different zstd versions" — addressed the dictionary case in mixed-version deployments.

A related question was recently filed as #4705 (v1.4.1 ↔ v1.5.7 in a RocksDB context). Our scenario is similar in shape but narrower in version span and specific to the raw-content prefix dictionary API, so filing separately for clarity.

Given those guarantees, we would like a targeted confirmation for our exact API shape before rolling a version bump into a large production deployment.

Scenario

We use zstd for block-level compression in a distributed service, statically linked into each node. We are upgrading from libzstd 1.5.5 to 1.5.7 as a rolling deployment. During the
rolling window (and as a rollback safety net) nodes on either version must be able to decode data produced by the other.

Encoder-side call shape (both versions)

- Single-shot buffer-to-buffer only — no streaming APIs.
- ZSTD_compress_usingDict(cctx, dst, dstCap, src, srcSize, dict, dictSize, level=6)
- ZSTD_decompress_usingDict(dctx, dst, dstCap, src, srcSize, dict, dictSize)
- dict is a raw-content prefix (not a serialized zstd dictionary with the ZSTD_MAGIC_DICTIONARY header); no ZDICT_* calls; no ZSTD_CDict / ZSTD_DDict objects.
- No ZSTD_CCtx_setParameter calls anywhere — every advanced parameter is at library default. Specifically:
- ZSTD_c_format = ZSTD_f_zstd1 (default; standard magic)
- ZSTD_c_checksumFlag = default (off)
- ZSTD_c_contentSizeFlag = default (on; srcSize is passed)
- ZSTD_c_dictIDFlag = default (raw-content prefix ⇒ no dictID embedded)
- ZSTD_c_windowLog, strategy, ZSTD_c_enableLongDistanceMatching — all default for level 6.
- Compression level: fixed at 6.
- Contexts are per-call (ZSTD_createCCtx / ZSTD_freeCCtx), not cached.

Question

Under the above encoder-side configuration, can we rely on:
1. A v1.5.5 decoder correctly decompressing every frame produced by a v1.5.7 encoder, and
2. A v1.5.7 decoder correctly decompressing every frame produced by a v1.5.5 encoder,

for both dictionary-compressed and non-dictionary calls, provided the same raw-content prefix bytes are supplied on both sides?

Reading #3110, #3711, and #3802, our understanding is "yes" — 1.5.6 and 1.5.7 shipped compressor fixes and decoder-tolerance improvements but no wire-format changes. We would appreciate
a one-line confirmation for the record so we can attach it to our internal rollout decision document.

Thank you.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.