[CORE][VL] Use protobuf to pass metrics from C++ to Java
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
The current way to fetch metrics from Velox task to Java is in way complicated:
https://github.com/apache/incubator-gluten/blob/e436fa4899eee052a5ed019542d8ce5a4183ffd5/cpp/core/jni/JniWrapper.cc#L497-L536
https://github.com/apache/incubator-gluten/blob/main/backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala
https://github.com/apache/incubator-gluten/blob/main/backends-clickhouse/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala
The long array stores a traversal of the metric tree Velox returns, then we actually tend to reconstruct the tree from the traversal array in Java side.
To simplify code, we can store the metrics returned by Velox directly in a structured object (perhaps, using protobuf), then perform Velox-to-Gluten metrics conversion from Java side in one go.
Contributor guide
Assessment
This issue has not been assessed yet.