JuliaIO / JuliaIO/TranscodingStreams.jl
Feature request: `Async` codec
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 90
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
As far as I can tell, TranscodingSteams (TS) is currently mostly used for de/compression codecs, which are quite CPU-intensive. In this case it would be beneficial to allow the de/compression to happen in a separate thread concurrently.
I think this can be solved by having an Async codec that wraps any TS, such that the underlying stream's transcoding happens in another thread. More specifically, Async could carry an internal buffer B along with its input (I) and output (O) buffers. When O runs dry, it will copy data from B to O, then schedule the transcoding from I to B.
I believe this codec should be in TS and not a separate package because:
- It's near-universally useful for users of TS
- The
Asynccodec might have to reach into TS's internals for efficiency, to make sure the underlying TS really does run its transcoding in a different task.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the TranscodingStreams codec interface and the existing decompression and compression codec behavior. Define how an Async wrapper would coordinate its input, internal buffer, and output while ensuring transcoding runs in another task. Done means the proposed codec works with arbitrary TranscodingStreams codecs and preserves the expected stream behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100