apache / apache/datafusion-comet

[Doc] decode does not appear in auto-generated compatibility docs

Open
#4,466 1 comment 0 reactions 0 assignees View on GitHub
area:expressions documentation priority:low
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

## Describe the bug

`decode` is handled in Comet via `CommonStringExprs.stringDecode` called from `versionSpecificExprToProtoInternal` in the per-Spark-version `CometExprShim`. There is no `CometExpressionSerde[StringDecode]` (or equivalent) registered in `QueryPlanSerde.scala`'s `stringExpressions` map.

Consequence: `decode` never appears in the auto-generated `docs/source/user-guide/compatibility.md`, neither under "incompatible" nor "unsupported" reasons. Users have no docs-side way to discover what Comet supports for `decode` (only literal `'utf-8'` charset, case-insensitive) or what it doesn't (every other charset, all four-arg Spark 4.0 cases described in #4465).

Surfaced by the string-expressions audit in apache/datafusion-comet#4461.

## Expected behavior

Register a `CometExpressionSerde[StringDecode]` (or a per-version equivalent that the shims dispatch to) so the doc generator picks the expression up and includes both the supported charset restriction and the legacy-flag gap as `getUnsupportedReasons()` entries.

## Additional context

- Shim: `spark/src/main/spark-{3.4,3.5,4.0,...}/org/apache/comet/shims/CometExprShim.scala`
- Helper: `CommonStringExprs.stringDecode` in `spark/src/main/scala/org/apache/comet/serde/strings.scala`
- Doc generator: `GenerateDocs` reads `get*Reasons()` from registered `CometExpressionSerde` instances; helpers called from shims are invisible to it.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.