haskell / haskell/binary

Provide buffer-agnostic interface for the Get monad.

Open
#120 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
120
Forks
70
PR merge metrics
No merged PRs in 30d

Description

Despite the fact that binary was primary designed with lazy bytestrins in mind, it also very useful for generic serialization with various types of buffers. And it works quite well for Put monad: we have `execPut :: PutM a -> Builder` latter we can run this builder with `runBuilder :: Builder -> BufferWriter` from `Data.ByteString.Builder.Extra` on any type of buffer: the interface requires vanilla pointer of type `Ptr Word8` and also provides information about minimal buffer size and number of bytes written to buffer. This function can be used to generate lazy bytestring, to write directly to device buffer, I use this function to write data to preallocated memory buffer in my application and then to generate strict bytestring from it.

Unfortunately similar functionality for Get monad is missed from the package. The closest function we have is `runGetIncremental :: Get a -> Decoder a` but it still operates on ByteStrings (not on pointers) and it even had serious performance issue until https://github.com/kolmodin/binary/commit/7532daa8789e5199109bb1fcde367d71effb07e2 Even been fixed, it still introduces some overhead by relatively complex handling logic, and this can be noticeable effect on very small Getters (in my application I have a getter which reads 6 bytes, even very small overhead on it affects performance).

I think Decoder should be redesigned to operate on pointers and buffer size information in similar way as `Builder` from the `Data.ByteString` does. Having such interface it would be easy to create ByteString version, as well as any custom buffer reader with minimal overhead.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing runGetIncremental :: Get a -> Decoder a interface and compare it with execPut, Builder, and runBuilder from Data.ByteString.Builder.Extra. Done means Decoder can operate on pointers and buffer-size information, while supporting a ByteString implementation and custom buffer readers with less overhead.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.