vectordotdev / vectordotdev/vector
Length-delimited framing actually emitting big-endian length, not little-endian
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
The docs for framing in sinks say:
length_delimited
Event data is prefixed with its length in bytes.
The prefix is a 32-bit unsigned integer, little endian.
Where in fact this framing emits Big Endian-encoded integer.
Documentation needs to be fixed.
See here, the tokio_util::codec::LengthDelimitedCodec codec is used:
https://github.com/vectordotdev/vector/blob/7cf2f009dbd9be4177dfbce7950cd82d57f93448/lib/codecs/src/encoding/framing/length_delimited.rs#L25
And this codec emits big-endian: https://docs.rs/tokio-util/latest/tokio_util/codec/length_delimited/#getting-started
The returned transport implements Sink + Stream for BytesMut. It encodes the frame with a big-endian u32 header denoting the frame payload length:
I've confirmed it when I was writing a server to receive Vector events.
Configuration
socket:
type: socket
inputs:
- parse_logs
mode: unix
path: /tmp/foo.socket
encoding:
codec: text
framing:
method: length_delimited
healthcheck:
enabled: false
Version
0.35
Debug Output
No response
Example Data
No response
Additional Context
No response
References
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Search the repository documentation for the length_delimited framing text under sinks, then compare it with the referenced lib/codecs/src/encoding/framing/length_delimited.rs implementation and the tokio-util codec documentation. Done means the documented byte order matches the big-endian framing actually emitted by LengthDelimitedCodec.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 62/100