AppendableRecordBatch
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
At Space and Time, we would like to make an OLTP columnar DB based on Arrow (see noisepage for precedent). Unfortunately, Arrow RecordBatches are immutable by design.
**Describe the solution you'd like**
Fortunately however, append-only and immutable are not necessarily at odds, as long as one takes immutable RecordBatch snapshots at a point-in-time of the append-only version. See [this gist](https://gist.github.com/avantgardnerio/60a53a3481f3d7844efa12346ef6f814) for a working PoC implementation:

**Describe alternatives you've considered**
1. using arrow2
2. trying some trickery with making the `MutableBuffer` work, vs preallocating ones from the immutable package
Contributor guide
Research direction
Start by reviewing the linked working PoC and the existing Arrow RecordBatch, MutableBuffer, and immutable-package APIs. Establish the required append-only snapshot semantics and tests before implementation; done means the project has an agreed scope for supporting appendable batches without violating immutable snapshots.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100