Bug triage results: 2026-08-17

Triage pass over the open `requires-triage` queue, per the project [Bug Triage Guide](https://github.com/apache/datafusion-comet/blob/main/docs/source/contributor-guide/bug_triage.md).

- Date: 2026-08-17
- Total issues processed: 33 (31 triaged, 2 skipped, 0 failed)
- Type counts: 8 bugs, 23 enhancements
- Priority counts applied: `priority:critical` 2, `priority:high` 0, `priority:medium` 3, `priority:low` 3
- Guide: [docs/source/contributor-guide/bug_triage.md](https://github.com/apache/datafusion-comet/blob/main/docs/source/contributor-guide/bug_triage.md)

Labels have already been applied. A reviewer should spot-check the calls below and close this issue when satisfied; corrections should be made directly on the affected issue.

Notes on label availability and pre-existing labels:

- The guide lists `spark 4` as a pre-existing area indicator, but the repository only has `spark 4.0` / `spark 4.1` / `spark 4.2` / `spark 3.x`, so no `spark 4` label was applied in this pass.
- `area:Iceberg` exists in the repository but is not in the guide's area table, so it was not applied to the Iceberg sort-merge follow-ups (#5337–#5343); `area:scan` was used instead. Worth deciding whether the guide should add it.
- Where an issue already carried non-guide labels (`correctness`, `performance`, `test`, `array expressions`, `temporal expressions`, `user experience`), those were left untouched and are included in the area lists below for context.
- Two enhancements arrived with a priority label already set by the reporter: #5383 (`priority:medium`) and #5346 (`priority:low`). The guide says only bugs receive a priority label. This pass did not add priority labels to enhancements and did not strip the pre-existing ones; a reviewer may want to remove them.

## Bugs

### priority:critical

- to_time / try_to_time: native parser rejects 'T12' and '12:30:45.' which Spark accepts ([#5366](https://github.com/apache/datafusion-comet/issues/5366))
- Area labels: `area:expressions`, `correctness`, `temporal expressions`
- Rationale: `try_to_time` returns `NULL` where Spark returns a value, and fractional digits 7–9 are truncated into a different stored nanosecond value, both with no error surfaced — the guide's "silent wrong results" case at step 1 of the decision tree. Escalated from the reporter's `priority:medium` (see escalations below).
- Boolean-to-decimal cast produces invalid Decimal128 when 10^scale does not fit precision ([#5334](https://github.com/apache/datafusion-comet/issues/5334))
- Area labels: `area:expressions`
- Rationale: `cast_boolean_to_decimal` returns `Ok` with an unscaled value outside the declared precision because `with_precision_and_scale` validates only type metadata, so Comet emits invalid decimal data as success where Spark errors or returns `NULL` — data corruption, `priority:critical` per the guide's first decision-tree question.

### priority:medium

- Report native child-operator spill metrics in Spark task metrics for unified shuffle plans ([#5382](https://github.com/apache/datafusion-comet/issues/5382))
- Area labels: `area:shuffle`
- Rationale: the task-metric bridge publishes spill counters only from the root `ShuffleWriterExec` node, so a task that spills 160 MiB reports 32 MiB — a functional defect in existing reporting rather than a new feature, and diagnostic-only, so `priority:medium` per step 3. Classified consistently with #5336, the same category of metric defect.
- Task input metrics are unreliable when a native block mixes a native scan with a JVM input ([#5336](https://github.com/apache/datafusion-comet/issues/5336))
- Area labels: `area:scan`
- Rationale: the completion listener is registered after the iterator's close listener, so on the busy-poll path the reported `bytes_scanned` is stale or zero; reporter's `priority:medium` confirmed — broken feature, no query-result impact.
- `NegativeExpr::get_properties` reports child ordering and range unchanged ([#5330](https://github.com/apache/datafusion-comet/issues/5330))
- Area labels: `area:expressions`
- Rationale: `with_order` is an identity function as used, so `-a` is reported with the child's unflipped ordering and un-negated range; the defect is demonstrated at unit level with no end-to-end query yet, so `priority:medium` per step 3 rather than critical (see escalations below).

### priority:low

- Nine expression benchmark rows labelled "Comet" are measuring Spark ([#5372](https://github.com/apache/datafusion-comet/issues/5372))
- Area labels: `area:expressions`, `test`, `performance`
- Rationale: the deliverable is corrected benchmark reporting, which the guide's step 4 places at `priority:low` as a tooling/test-only issue; the underlying `ShortType` fallback may be more than that (see escalations below).
- ANSI `abs` integer overflow errors use Arrow-style type names instead of Spark's ([#5356](https://github.com/apache/datafusion-comet/issues/5356))
- Area labels: `area:expressions`
- Rationale: the error class matches Spark (`ARITHMETIC_OVERFLOW`) and only the interpolated type name differs (`Int64` vs `long`), so nothing is broken beyond user-visible message text — cosmetic, step 4.
- Improve CI to run enough tests when dependency upgraded ([#5333](https://github.com/apache/datafusion-comet/issues/5333))
- Area labels: `area:ci`
- Rationale: kept as `bug` because the guide counts CI failures as bugs and this gap let #5289 break `main`, but priority is `priority:low` per step 4, whose examples are explicitly CI and build tooling; the breakage itself has been reverted (#5332).

## Enhancements

- Reduce JNI round-trips in the unified memory pools (batching, hysteresis, cheaper call path) ([#5383](https://github.com/apache/datafusion-comet/issues/5383))
- Area labels: `area:ffi`, `performance`
- Rationale: a performance optimization of a working code path — chunked acquire with retained slack and hysteresis — which the guide's type table places under `enhancement`.
- Make native contrib scan dispatch generic (remove per-contrib arms from core Rust) ([#5379](https://github.com/apache/datafusion-comet/issues/5379))
- Area labels: `area:scan`
- Rationale: source-level tidiness, replacing `#[cfg]`-gated per-contrib calls with a `type_url -> handler` registry; the issue itself notes a default build links zero contrib symbols, so nothing is broken.
- Move contrib scan proto messages out of core operator.proto into contrib/<name>/proto ([#5378](https://github.com/apache/datafusion-comet/issues/5378))
- Area labels: `area:scan`
- Rationale: a refactor that needs new per-contrib proto build infrastructure; the messages are inert in core today and the relocation is invisible on the wire, so there is no defect.
- Use end to end microbenchmarks to prevent performance regressions between releases ([#5373](https://github.com/apache/datafusion-comet/issues/5373))
- Area labels: `area:ci`
- Rationale: asks for benchmark documentation, a reproducible EC2 instance type, and eventual scheduled automation — new process and tooling, not a defect.
- Investigate: scan + shuffle-write slower than Spark on deeply nested schemas ([#5355](https://github.com/apache/datafusion-comet/issues/5355))
- Area labels: `area:scan`, `area:shuffle`, `performance`
- Rationale: classified as an enhancement because it is an explicitly unverified collection of static-analysis optimization candidates ("no benchmark numbers yet") rather than a confirmed regression from a prior Comet release; if a benchmark confirms Comet is slower than Spark on a real workload, this should be re-triaged as a `priority:medium` performance bug.
- Run replace natively by default when the search string is a non-empty literal ([#5354](https://github.com/apache/datafusion-comet/issues/5354))
- Area labels: `area:expressions`, `performance`
- Rationale: the current dispatcher path is correct; this asks to widen the native default for a provably safe subset, which is a performance optimization.
- Native candidate assessment: upper/lower deferred on the lack of a per-batch defer-to-dispatcher mechanism ([#5353](https://github.com/apache/datafusion-comet/issues/5353))
- Area labels: `area:expressions`, `performance`
- Rationale: records a missing capability — plan-time-only native/dispatcher selection with no way to defer a single batch — as a blocker for a class of optimizations; nothing behaves incorrectly today.
- Support Apache Celeborn remote shuffle service ([#5352](https://github.com/apache/datafusion-comet/issues/5352))
- Area labels: `area:shuffle`
- Rationale: new functionality that Comet does not have today, the guide's canonical `enhancement` case.
- Run rlike natively by default for patterns that are provably Java-regex equivalent ([#5351](https://github.com/apache/datafusion-comet/issues/5351))
- Area labels: `area:expressions`, `performance`
- Rationale: proposes a plan-time pattern analyzer so a safe subset can default to the existing native kernel; the conservative current behavior is not a defect.
- Implement sequence natively for integral types instead of JVM codegen dispatch ([#5349](https://github.com/apache/datafusion-comet/issues/5349))
- Area labels: `area:expressions`, `array expressions`, `performance`
- Rationale: the codegen dispatcher is correct; this asks for a native implementation to remove per-row allocations, so it is a performance optimization.
- Implement unbase64 natively instead of JVM codegen dispatch ([#5347](https://github.com/apache/datafusion-comet/issues/5347))
- Area labels: `area:expressions`, `performance`
- Rationale: `unbase64` is already `Compatible()` via codegen dispatch, so behavior matches Spark; the ask is a new native kernel for the decode direction.
- Tag fallback reasons with structured values instead of pre-formatted strings ([#5346](https://github.com/apache/datafusion-comet/issues/5346))
- Area labels: none (`user experience`)
- Rationale: a refactor of the fallback-reason representation across ~205 call sites to make reasons dedupable and queryable — an improvement to existing working reporting.
- Bound memory when a partition has many files ([#5343](https://github.com/apache/datafusion-comet/issues/5343))
- Area labels: `area:scan`
- Rationale: a scoped follow-up to #5323 proposing bounded-concurrency file reads in the sorted merge path; filed as planned work on a feature that just landed, not a reported failure.
- Broaden storage-partitioned-join tests, then enable reportPartitioning by default ([#5342](https://github.com/apache/datafusion-comet/issues/5342))
- Area labels: `area:scan`
- Rationale: adds test coverage as the precondition for flipping a config default — new coverage plus a behavior change, no existing defect.
- Skip the merge wrapper for single-file partitions ([#5341](https://github.com/apache/datafusion-comet/issues/5341))
- Area labels: `area:scan`
- Rationale: avoids a no-op `SortPreservingMergeExec` for single-file partitions; correct today, just slightly wasteful, so a performance optimization.
- Keep read concurrency in the sorted merge path ([#5340](https://github.com/apache/datafusion-comet/issues/5340))
- Area labels: `area:scan`
- Rationale: asks to benchmark the many-small-files case and add read-ahead if it regresses — prospective optimization work, with the regression not yet established.
- Support transformed sort orders (bucket / truncate / etc.) ([#5339](https://github.com/apache/datafusion-comet/issues/5339))
- Area labels: `area:scan`
- Rationale: extends ordering reporting from identity sort keys to Iceberg transforms; the current narrow scope is documented as deliberate and always correct.
- Merge on sort keys even when they are not selected ([#5338](https://github.com/apache/datafusion-comet/issues/5338))
- Area labels: `area:scan`
- Rationale: widens when the sorted merge can be used by reading unprojected sort columns; skipping the merge today is a correct-but-conservative fallback.
- Sort-aware Iceberg reads: per-partition streaming merge + report ordering and grouping (v1) ([#5337](https://github.com/apache/datafusion-comet/issues/5337))
- Area labels: `area:scan`
- Rationale: the umbrella issue for a new capability — preserving Iceberg sort order and reporting key-grouped partitioning — with #5338–#5343 as its scoped follow-ups.
- Add mode to run Comet planning but execute with Spark, so users can assess potential Comet coverage ([#5335](https://github.com/apache/datafusion-comet/issues/5335))
- Area labels: none
- Rationale: proposes a new `spark.comet.planOnly.enabled` dry-run mode so users can estimate coverage without execution risk — new functionality and a new config.
- Add a fallback-invariance suite: query outcome should not change when a compatible expression is forced from native to Spark ([#5328](https://github.com/apache/datafusion-comet/issues/5328))
- Area labels: `area:expressions`
- Rationale: new test infrastructure sweeping the 293-expression registry for an invariant, replacing four hand-picked per-expression tests; classified as an enhancement even though it is expected to surface bugs, because the ask itself is coverage.
- Comet 1.1.0 Release (September) ([#5327](https://github.com/apache/datafusion-comet/issues/5327))
- Area labels: none
- Rationale: a release-planning tracking issue; `enhancement` is the closer of the two type labels since nothing is reported broken.
- Could Rust 1.98 new floating point optimizations help Comet? ([#5326](https://github.com/apache/datafusion-comet/issues/5326))
- Area labels: none
- Rationale: an open question about adopting new compiler float-math optimizations — exploratory performance work, no defect described.

## Escalations to consider

- to_time / try_to_time: native parser rejects 'T12' and '12:30:45.' which Spark accepts ([#5366](https://github.com/apache/datafusion-comet/issues/5366))
- Escalated from the reporter's `priority:medium` to `priority:critical`. The `to_time` cases raise a visible error, which alone would sit at `priority:medium`, but `try_to_time` returns `NULL` for input Spark parses successfully and case 3 stores a different nanosecond value — both silent, which the guide's step 1 and the "correctness over crashes" principle put at critical. Narrow surface (Spark 4.1 only, unusual literals) is the counter-argument, but the guide's own critical examples include comparably narrow cast edge cases. Revert to `priority:medium` if the reviewer judges case 3 unobservable and the `try_to_time` `NULL` acceptable.
- `NegativeExpr::get_properties` reports child ordering and range unchanged ([#5330](https://github.com/apache/datafusion-comet/issues/5330))
- Left at `priority:medium`, but this is the escalation trigger to watch: the wrong ordering is reported to the optimizer, so if a plan can be shown where DataFusion drops or keeps a sort based on it, the result is silently mis-ordered output and this becomes `priority:critical`. The issue currently demonstrates the defect only through an expression-property unit test.
- Nine expression benchmark rows labelled "Comet" are measuring Spark ([#5372](https://github.com/apache/datafusion-comet/issues/5372))
- Left at `priority:low` as a benchmark-reporting defect, but the issue notes that Comet does support short-to-int and short-to-long casts, so if the `ShortType` projection failing to be replaced reproduces outside `CometCastNumericToNumericBenchmark`, it is a real coverage gap affecting user queries and warrants `priority:medium`.
- Improve CI to run enough tests when dependency upgraded ([#5333](https://github.com/apache/datafusion-comet/issues/5333))
- Left at `priority:low`. The guide's escalation trigger — "a `priority:low` CI flake is blocking PR merges consistently" — applies if dependency bumps break `main` again; #5289 is one occurrence and was reverted.

## Skipped — needs more info

- Bug triage results: 2026-08-11 ([#5325](https://github.com/apache/datafusion-comet/issues/5325))
- Not a bug report or a feature request but the summary issue from the previous triage pass, so neither type label applies and no priority is meaningful. `requires-triage` was left in place because the skill only removes it alongside a type label. The reviewer should close it once the 2026-08-11 calls have been spot-checked; it will otherwise reappear in every future pass.
- Bug triage results: 2026-08-03 ([#5231](https://github.com/apache/datafusion-comet/issues/5231))
- Same as above, for the 2026-08-03 pass. This one has now been open across three triage passes, so it is worth either closing it or deciding that triage summary issues should be exempted from the `requires-triage` auto-label.

Start with docs/source/contributor-guide/bug_triage.md and spot-check the 31 linked issues against the recorded labels and rationales. Confirm the noted label discrepancies and close #5385 when satisfied; corrections belong on the affected issues.

Written by the indexing model from the issue text.