vectordotdev / vectordotdev/vector
Improve partial message detection heuristics at docker source
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
From the k8s PR:
Partial messages are only split in docker at the maximum message length. Thus, for a message to be partial it also has to have a certain length. Now, whether that message will or won't actually be partial if it has exactly the max length is unknown. We probably should consider all messages with the exact length of 16 * 1024 bytes partial by-default, and then, if they end with newline - consider that an exception and make them non-partial. Logic is still broken, but better than what we currently have.
Docker source is affected too.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Docker source's partial-message detection and compare it with the referenced Kubernetes PR. Done means messages exactly 16 * 1024 bytes are treated as partial by default, except when they end with a newline; run the Docker source tests, if present, to verify both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100