apache / apache/datafusion-comet
[Doc] CAST has no explicit TimeType branch (Spark 4.1)
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 375
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 221
Description
## Describe the bug
Spark 4.1 introduced `TimeType` (HH:mm:ss). Spark 4.1's `Cast.scala` adds many `TimeType` arms (`StringType <-> TimeType`, `TimeType <-> TimeType`, `TimeType -> IntegralType / DecimalType`, etc.). `CometCast.isSupported` has no `TimeType` arm at all, so every TimeType-involving cast falls into the default `case _ => unsupported(fromType, toType)` and falls back to Spark.
The fallback is correct, but it is implicit, and `getUnsupportedReasons()` on `CometCast` does not enumerate `TimeType` as one of the unsupported branches, so the auto-generated compatibility guide doesn't mention it.
Surfaced by the cast audit (collection PR queue).
## Expected behavior
Either:
1. Add an explicit `TimeType` arm to `CometCast.isSupported` that returns `Unsupported(Some("Cast to/from TimeType is not yet supported"))`, OR
2. Implement native `TimeType` cast (depends on `TimeType` being supported broadly across Comet, tracked by the broader date/time epic #4418).
The minimum here is the explicit `Unsupported` arm so the compatibility guide reflects it.
## Additional context
- Spark reference: `Cast.scala` in `/tmp/spark-v4.1.1/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/`.
- Comet matrix: `CometCast.scala`.
- Related: #4418 (EPIC for date/time expressions on Spark 4.x).
Contributor guide
Research direction
Start in CometCast.scala and compare its cast handling with Cast.scala from the referenced Spark 4.1.1 source. Trace how unsupported reasons feed the auto-generated compatibility guide, then verify that the guide explicitly reports casts involving TimeType as unsupported without changing native cast behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100