nodejs / nodejs/node

zstd decoding complete frame in read stream halts stream

Abierto
#64,741 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

stream zlib
Lenguaje dominante
JavaScript
Estrellas
122k
Forks
37.3k
Merge medio
4 d 2 h
PR fusionados (30 d)
283

Descripción

Version

v26.5.0

Platform
Darwin Mac.lan 24.6.0 Darwin Kernel Version 24.6.0: Tue Apr 21 20:19:12 PDT 2026; root:xnu-11417.140.69.710.16~1/RELEASE_ARM64_T6041 arm64
Subsystem

No response

What steps will reproduce the bug?
import { Buffer } from 'node:buffer';
import * as consumers from 'node:stream/consumers';
import * as zlib from 'node:zlib';

function consume(codec, writes) {
	const inflate = codec.createDecompress();
	const result = consumers.buffer(inflate);
	writes.forEach(write => inflate.write(write));
	inflate.end();
	return result;
}

function describe(buffer) {
	const aa = buffer.includes(0x41);
	const bb = buffer.includes(0x42);
	const which = function() {
		if (aa && bb) {
			return 'frameA + frameB';
		} else if (aa) {
			return 'frameA only';
		} else if (bb) {
			return 'frameB only';
		} else {
			return 'nothing';
		}
	}();
	return `${buffer.length} bytes (${which})`;
}

async function attempt(codec, writes) {
	try {
		return describe(await consume(codec, writes));
	} catch (error) {
		return `threw: ${error.message}`;
	}
}

const tests = [ {
	name: 'zstd',
	compress: buffer => zlib.zstdCompressSync(buffer),
	createDecompress: () => zlib.createZstdDecompress(),
}, {
	name: 'gzip',
	compress: buffer => zlib.gzipSync(buffer),
	createDecompress: () => zlib.createGunzip(),
} ];

console.log(`node ${process.version}`);
for (const codec of tests) {
	// 'A' / 'B'
	const frameA = codec.compress(Buffer.alloc(2000, 0x41));
	const frameB = codec.compress(Buffer.alloc(2000, 0x42));
	const concatenated = Buffer.concat([ frameA, frameB ]);

	// BUG: both frames in a single write — the boundary between them falls inside one input buffer.
	const bug = await attempt(codec, [ concatenated ]);
	// COUNTER-EXAMPLE: the same two frames, one per write — the boundary lands at a write boundary.
	const counter = await attempt(codec, [ frameA, frameB ]);

	console.log(`${codec.name}:`);
	console.log(`  stream, both frames in ONE write   -> ${bug}`);
	console.log(`  stream, one frame per write        -> ${counter}`);
}
How often does it reproduce? Is there a required condition?

This reproduces every time

What is the expected behavior? Why is that the expected behavior?

Decoding concatenated zstd payloads should decode correctly as one stream. Indeed, zstdcat will decode this correctly (omitted from example but you can take my word for it).

What do you see instead?
marcel[1:42:21PM] [~/xx] ~/Downloads/node-v26.5.0-darwin-arm64/bin/node a.mjs
node v26.5.0
zstd:
  stream, both frames in ONE write   -> 2000 bytes (frameA only)
  stream, one frame per write        -> 4000 bytes (frameA + frameB)
gzip:
  stream, both frames in ONE write   -> 4000 bytes (frameA + frameB)
  stream, one frame per write        -> 4000 bytes (frameA + frameB)
Additional information

This will cause spurious failures while decoding zstd streams. The odds of it happening are a function of stream read size vs zstd chunk size. In my case I ran into this while decoding a stream after about 3GB and tracked it down to this bug.

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Empieza ejecutando el script de reproducción contra las API de streaming de zlib y compara frames zstd concatenados en una sola escritura con un frame por escritura. Rastrea el manejo del stream de descompresión zstd de Node.js y añade una prueba de regresión para el caso de una sola escritura; se considera terminado cuando ambos frames producen 4000 bytes, coincidiendo con el comportamiento de gzip y zstdcat.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
javascript, nodejs
Área
stream-processing
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Tranquilo
Claridad
Bastante claro
Aptitud para principiantes
68/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.