zstd decoding complete frame in read stream halts stream
Nadie ha tomado este issue todavía.
- Lenguaje dominante
- JavaScript
- Estrellas
- 122k
- Forks
- 37.3k
- Merge medio
- 4 d 2 h
- PR fusionados (30 d)
- 283
Descripción
Version
v26.5.0
Platform
Darwin Mac.lan 24.6.0 Darwin Kernel Version 24.6.0: Tue Apr 21 20:19:12 PDT 2026; root:xnu-11417.140.69.710.16~1/RELEASE_ARM64_T6041 arm64
Subsystem
No response
What steps will reproduce the bug?
import { Buffer } from 'node:buffer';
import * as consumers from 'node:stream/consumers';
import * as zlib from 'node:zlib';
function consume(codec, writes) {
const inflate = codec.createDecompress();
const result = consumers.buffer(inflate);
writes.forEach(write => inflate.write(write));
inflate.end();
return result;
}
function describe(buffer) {
const aa = buffer.includes(0x41);
const bb = buffer.includes(0x42);
const which = function() {
if (aa && bb) {
return 'frameA + frameB';
} else if (aa) {
return 'frameA only';
} else if (bb) {
return 'frameB only';
} else {
return 'nothing';
}
}();
return `${buffer.length} bytes (${which})`;
}
async function attempt(codec, writes) {
try {
return describe(await consume(codec, writes));
} catch (error) {
return `threw: ${error.message}`;
}
}
const tests = [ {
name: 'zstd',
compress: buffer => zlib.zstdCompressSync(buffer),
createDecompress: () => zlib.createZstdDecompress(),
}, {
name: 'gzip',
compress: buffer => zlib.gzipSync(buffer),
createDecompress: () => zlib.createGunzip(),
} ];
console.log(`node ${process.version}`);
for (const codec of tests) {
// 'A' / 'B'
const frameA = codec.compress(Buffer.alloc(2000, 0x41));
const frameB = codec.compress(Buffer.alloc(2000, 0x42));
const concatenated = Buffer.concat([ frameA, frameB ]);
// BUG: both frames in a single write — the boundary between them falls inside one input buffer.
const bug = await attempt(codec, [ concatenated ]);
// COUNTER-EXAMPLE: the same two frames, one per write — the boundary lands at a write boundary.
const counter = await attempt(codec, [ frameA, frameB ]);
console.log(`${codec.name}:`);
console.log(` stream, both frames in ONE write -> ${bug}`);
console.log(` stream, one frame per write -> ${counter}`);
}
How often does it reproduce? Is there a required condition?
This reproduces every time
What is the expected behavior? Why is that the expected behavior?
Decoding concatenated zstd payloads should decode correctly as one stream. Indeed, zstdcat will decode this correctly (omitted from example but you can take my word for it).
What do you see instead?
marcel[1:42:21PM] [~/xx] ~/Downloads/node-v26.5.0-darwin-arm64/bin/node a.mjs
node v26.5.0
zstd:
stream, both frames in ONE write -> 2000 bytes (frameA only)
stream, one frame per write -> 4000 bytes (frameA + frameB)
gzip:
stream, both frames in ONE write -> 4000 bytes (frameA + frameB)
stream, one frame per write -> 4000 bytes (frameA + frameB)
Additional information
This will cause spurious failures while decoding zstd streams. The odds of it happening are a function of stream read size vs zstd chunk size. In my case I ran into this while decoding a stream after about 3GB and tracked it down to this bug.
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Línea de trabajo
Empieza ejecutando el script de reproducción contra las API de streaming de zlib y compara frames zstd concatenados en una sola escritura con un frame por escritura. Rastrea el manejo del stream de descompresión zstd de Node.js y añade una prueba de regresión para el caso de una sola escritura; se considera terminado cuando ambos frames producen 4000 bytes, coincidiendo con el comportamiento de gzip y zstdcat.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- javascript, nodejs
- Área
- stream-processing
- Tipo de issue
- Error
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Estado de actividad
- Tranquilo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 68/100