Documentation for how to use IPC RecordBatch compression
- Dominant language
- TypeScript
- Stars
- 112
- Forks
- 23
- Avg merge
- 21h 18m
- Merged PRs (30d)
- 8
Description
I think I understand this after reading through the code, but it would be great to have some documentation included in the repo about how to use this feature. Documentation is not a strength of Arrow JS.
- If you call `registerCompressionCodecs` and no codecs have been set, then it'll automatically register `lz4js` and fetch `zstd-codec`? That might be unexpected, especially since `zstd-codec` is probably pretty large, right? To turn that off, you have to set the `compressionRegistry` for ZSTD to anything other than `null`? And then if set to something "fake", you just have to make sure you don't try to load ZSTD files?
- lz4js is always bundled with arrow-js now it looks like? So there's no penalty to always registering lz4 compression. But there is a penalty to registering zstd compression because that'll trigger the fetch of the codec, right?
- Oh wait, that `registerCompressionCodecs` is just in the tests..? So users always have to copy that implementation? That's fine, just trying to understand.
_Originally posted by @kylebarron in https://github.com/apache/arrow-js/issues/14#issuecomment-3358143312_
Contributor guide
Assessment
This issue has not been assessed yet.