matrixorigin / matrixorigin/matrixone

[Refactoring]: Batch refactoring

Open
#23,722 0 comments 0 reactions 1 assignee Claimed by @fengttt View on GitHub
kind/refactor needs-triage
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

### Why do you want to refactor this code?

```Markdown
Batch is a container of vectors. It should hold NOTHING more than data.
At this moment, there are a few things in the batch,

Recursive -- this should be maintained by operator.
ShuffleIdx -- this should be maintianed by operator.

Attrs -- this is also questionable. The attrs of a batch should be known at compile time, therefore maintained by operator

Vecs -- this one is useful,

ExtraBuf -- Agg function use this to pass state, this is at least a bad practice, if all agg functions maintain their state as vectors, we should just use Vecs to pass state around, this will save quite some ser/deser code. But the new Agg framework already put the ser/deser code in just one place, so, this one is more or less OK.

rowCount, need to keep

offHeap -- this one should be useless, batch generator should keep track of data lifecycle so this one is at best redundant, possibly serious bug.
```

### Describe the solution you'd like

Clean up code, remove dead/buggy redundant code.

### Describe alternatives you've considered

_No response_

### Additional information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.