Avoid per-write virtual dispatch in `DictionaryValuesWriter.shouldFallBack()` by caching the size-exceeded check
- 主要语言
- Java
- 星标
- 3.1k
- 派生
- 1.6k
- 平均合并
- 3 天 12 小时
- 30 天内合并 PR
- 33
描述
### Describe the enhancement requested
`DictionaryValuesWriter.shouldFallBack()` is called by `FallbackValuesWriter.checkFallback()` after every single value write. The current implementation dispatches a virtual call to `getDictionarySize()` on every invocation:
```java
public boolean shouldFallBack() {
return dictionaryByteSize > maxDictionaryByteSize || getDictionarySize() > MAX_DICTIONARY_ENTRIES;
}
```
`getDictionarySize()` is an abstract method overridden in each typed subclass (Binary, Long, Double, Integer, Float) to return the backing map's `.size()`. Since `shouldFallBack()` is polled after every write, including writes of duplicate values that do not grow the dictionary, the virtual dispatch and map-size query are redundant work for the common case where most values are already in the dictionary.
Both `dictionaryByteSize` and the dictionary entry count can only increase when a new entry is added (inside the `if (id == -1)` branch of each subclass's write method). Therefore the size-exceeded condition can only transition from `false` to `true` at that exact point.
### Proposal
Replace the per-write check with a cached boolean `dictionarySizeExceeded` flag. Introduce a `checkDictionarySizeLimit(int newDictionarySize)` method that subclass write methods call only when a new dictionary entry is actually added. `shouldFallBack()` then returns the cached flag directly, a simple field read with no virtual dispatch.
### Component(s)
Core
贡献指南
这个仓库没有索引到贡献指南
调研方向
Start with DictionaryValuesWriter.shouldFallBack() and FallbackValuesWriter.checkFallback(), then inspect the Binary, Long, Double, Integer, and Float typed subclass write methods, especially their id == -1 branches. Confirm that the size-exceeded state is updated only when a new dictionary entry is added and that fallback behavior remains unchanged for duplicate writes.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- data-engineering
- Issue 类型
- 重构
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 冷清
- 描述清晰度
- 描述清楚
- 新手友好度
- 76/100