apache / apache/sedona-db

Pre-allocate memory for scalar functions more effectively

Open
#1,292 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
503
Forks
61
Avg merge
2d 4h
Merged PRs (30d)
90

Description

For most of our scalar functions we do something like this:

```rust
let mut builder = BinaryBuilder::with_capacity(
executor.num_iterations(),
WKB_MIN_PROBABLE_BYTES * executor.num_iterations(),
);
```

A lot of them, however, can do a better job pre-allocating the output based on previously available information (or potentially a cheap iteration over the values). For example, ST_Boundary output is usually similar to that of the input, and ST_Reverse is always exactly the same as the input.

It's possible this won't make much of a difference but there are at least some functions where this is trivial.

Contributor guide

Open the contributing guide

Research direction

Search the scalar-function implementations for BinaryBuilder::with_capacity, then compare ST_Boundary and ST_Reverse with their input values. Identify functions where output size is available or cheap to estimate, and consider the work done when those functions pre-allocate more accurately without changing results.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.