HdrHistogram / HdrHistogram/HdrHistogram_rust
Consider allowing serialization into a Vec<u8> directly
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 372
- Forks
- 50
- PR merge metrics
- No merged PRs in 30d
Description
Currently, serialization output is directed to a Write in both the V2 and V2 + DEFLATE serializers. This is a very flexible abstraction, but the serialization formats dictate a length prefix, which means that output must be buffered before writing to the writer so that we can drop in the final length in the appropriate spot.
This buffering is especially sad in the DEFLATE case, since we already have a Vec<u8> that will hold the uncompressed serialization, but we can only expose it to the underlying V2 serializer as a Write, so the uncompressed bytes get written to V2's buffer, then copied into V2 + DEFLATE's buffer, then compressed into another buffer, then written to the user-provided writer.
Some options:
- Functions for
WriteorVec<u8>, where the former buffers into an internalVec<u8> - Functions for
WriteorWrite + Seek. Need to benchmark this to see if it's measurably different from theVec<u8>case. If it isn't, this seems preferable as it is more general. - Give up on
Writeand only serialize into eitherWrite + Seekor aVec<u8>. Maybe supporting I/O streams that don't seek is a sufficiently niche case that it's not worth creating an easy path for? On one hand, I really want to encourage people to use these formats as a wire protocol in a monitoring system. On the other hand, maybe such a protocol would use a container format like protobuf around the serialized histogram anyway, and it's a waste of API complexity budget to support simpleWriteusage.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the V2 and V2 + DEFLATE serializers and tracing how their Write-based output is buffered. Benchmark the proposed Vec and Write + Seek approaches, then clarify which API design is preferred and what serialization behavior should be considered complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100