apache / apache/tsfile

Support Adaptive Encoding

Open
#902 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Java
Stars
203
Forks
104
Avg merge
2d 6h
Merged PRs (30d)
33

Description

Currently, TsFile uses a fixed encoding algorithm for each data type. For example, floating-point data may use Gorilla encoding.

However, data characteristics vary across datasets, and a fixed encoding algorithm may not always perform well. In some cases, it may even increase the encoded data size. We propose introducing an adaptive encoding mechanism that dynamically selects a more suitable encoding algorithm based on the actual data characteristics or encoding effectiveness.

Falling back to PLAIN when the current encoding is unsuitable can be one adaptive option, but it should not be the only one. The selection strategy, evaluation criteria, and switching granularity can be discussed separately.

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating TsFile’s fixed encoding path for each data type and reviewing the existing encoding implementations; the work is not ready to start until the selection strategy, evaluation criteria, switching granularity, and completion criteria are agreed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.