elastic / elastic/logstash

BufferedTokenizer may silently drop data when oversize input has no delimiters

Open
#18,321 3 comments 0 reactions 1 assignee Claimed by @andsel View on GitHub
bug status:confirmed
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

**Logstash information**:

Please include the following information:

1. Logstash version (e.g. `bin/logstash --version`): `9.2.0-SNAPSHOT`
2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker) _N/A_
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes) _N/A_

**Description of the problem including expected versus actual behavior**:

In https://github.com/elastic/logstash/pull/17229 changes to the buffered tokenizer introduce an edge-case where an oversized sequence of bytes that do _not_ include a delimiter can be silently dropped.

This occurs because size validation happens _after_ the token is fully accumulated, which does not occur if no trailing separator is detected.

**Steps to reproduce**:

Please include a *minimal* but *complete* recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

1.
2.
3.

**Provide logs (if relevant)**:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.