[VL] IncrementMetric doesn't work in some cases
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
The following test prints:
metric should be 3. Value: 0
metric should be 3. Value: 3
```
test("operation metrics - test gluten") {
withTempDir { tempDir =>
val dirPath = tempDir.getAbsolutePath
spark.range(0, 3)
.write
.format("delta")
.save(dirPath)
val metric1 = createMetric(spark.sparkContext, "number of rows")
val incrMetric1 = IncrementMetric(TrueLiteral, metric1)
spark.read.load(dirPath)
.select(col("_metadata.file_path"))
.filter(new Column(incrMetric1))
.collect()
println("metric should be 3. Value: " + metric1.value)
val metric2 = createMetric(spark.sparkContext, "number of rows")
val incrMetric2 = IncrementMetric(TrueLiteral, metric2)
spark.read.load(dirPath)
.select(input_file_name())
.filter(new Column(incrMetric2))
.collect()
println("metric should be 3. Value: " + metric2.value)
}
}
```
### Spark version
Spark-3.5.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Assessment
This issue has not been assessed yet.