ReplicaManager caches the projectionInfo to reduce the resource consumption for building projections.
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.
### Motivation
When we tested the column pruning capability of the log table, we found that there were some CPU bottlenecks on the server side. The characteristic of the table is that it has an extremely large number of columns, reaching tens of thousands. The CPU on the server side was fully utilized. After analyzing the flame graph, we discovered that the consumption at the data processing level was not significant. The main issue was that there was a large consumption when comparing some data types during the construction of the projection. The major part of the CPU overhead on the Fluss server side occurs in the stage of constructing the projection in FileLogProjection, among which the equals operations of various data types account for nearly 65%.
The `LogFileProjection` doesn't implement caching, which means that each `fetch` request requires reconstructing it, resulting in high costs. To solve this problem, a projection cache should be built in the `ReplicaManager`.
### Solution
_No response_
### Anything else?
_No response_
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.