matrixorigin / matrixorigin/matrixone

[Feature Request]: Fulltext index docid map

Open
#21,769 2 comments 0 reactions 1 assignee Claimed by @cpegeric View on GitHub
kind/feature priority/p0
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

### Is there an existing issue for the same feature request?

- [x] I have checked the existing issues.

### Is your feature request related to a problem?

```Markdown
Performance improvement for,

1. Really long fulltext key,
2. Update/delete of user data.
```

### Describe the feature you'd like

To support 1. really long primary key, 2. efficient update and delete, one idea is to add a docid map. Which is an index (also an internal table) that maps primary key to a docid (int64). docid is internally maintained, auto incr, and will get a new value when updated.

For update or delete to the user table, all we need to do is delete the docid map entry (and create a new entry for update). We could how many deletes are performed to this docid map and when it reaches a threshold we perform a merge/vacuum on the real fulltext table.

### Describe implementation you've considered

Fulltext index catalog entry can have a "version", or "option" of the index, with or without docid map.

With this entry we do not need to have upgrade step.

### Documentation, Adoption, Use Case, Migration Strategy

```Markdown

```

### Additional information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.