apache / apache/texera

Read the whole-number parts out of a timestamp column

Open
#8,478 0 comments 0 reactions 1 assignee Claimed by @kz930 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Task Summary

Aggregate groups by a column's values, and a timestamp has a different value in almost every row, so grouping by one puts each row in its own group. Nothing in Texera can make the coarser column that grouping wants: sales by month cannot be asked for, though the platform carries a TIMESTAMP type, a Gantt chart and a time-series plot.

An operator that reads the whole-number parts out of a timestamp — year, quarter, month, day, weekday, hour and the rest — closes that. It parses nothing: the column is already a moment by the time it arrives, and which text became which moment was settled upstream, the way KNIME, Alteryx and Spark all separate parsing from extraction.

The parts are read as ISO-8601 states them, since the engine and the exported Python do not agree by default: pandas counts Monday as 0 and `java.time` counts it as 1.

Not part of #8325: nothing here makes a workflow exportable. It does ship standalone code, so it comes after the trait #8327 introduces, and its parity assertions need the harness that issue builds.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.