RFC: Commit-Level Partition Min/Max EventTime Metadata in Apache Hudi

### Feature Description

**Summary**

This feature introduces **commit-level, partition-scoped mix/max event time metadata** in Apache Hudi.
Freshness metrics (e.g., min/max source event time) are recorded as commit metadata, allowing users and downstream systems to reason about **data freshness independently of commit time**.

Currently, _hoodie_commit_time only reflects when data was written, not when source data was produced. This is insufficient for freshness SLAs, debugging pipeline delays, or explaining partition-level freshness differences.

This proposal adds a **storage-layer primitive** for freshness tracking that is **engine-agnostic, backward-compatible, and opt-in**.

**Motivation**

Users need a reliable way to answer:

- How fresh is a specific partition?
- Which partitions are lagging and why?
- Is freshness degrading across derived tables?

Existing approaches rely on external systems or expensive scans and lack atomicity with Hudi commits.

**Design Overview**

- Freshness is observed during write, not inferred later
- Freshness is aggregated per destination partition
- Freshness is persisted atomically with the commit
- Hudi itself does not scan source tables or require query engine changes

**Freshness Signal Selection (Simplified Rule)**

Freshness metrics are selected using the **best available signal**, in strict order:

- Propagated freshness metadata from upstream Hudi commits (when present)
- Min/max aggregation of a configured event-time column during write
- No freshness metadata if neither signal is available

This avoids heuristic inference and works uniformly for raw and derived tables.

**Commit Metadata Extension**

Freshness metrics are stored under extraMetadata in the commit:

```
{
"extraMetadata": {
"hoodie.source.freshness": {
"partition=dt=2025-12-01": {
"min_event_time": "2025-12-01T10:28:00Z",
"max_event_time": "2025-12-01T10:31:00Z"
}
}
}
}
```

- Metadata is optional and additive
- Immutable once committed
- Ignored safely by older readers

### User Experience

**How users will use this feature**

This feature is **opt-in**. Existing pipelines continue to function unchanged.

Configuration Changes (Optional)
```
hoodie.source.freshness.enable=true
hoodie.source.freshness.event.time.field=ts
```

If disabled, no freshness metrics are collected or written.

**Behavior by Use Case**

**Raw ingestion tables (e.g., Kafka → Hudi)**

- Freshness computed from min/max of the configured event-time column

**Derived / transformed tables**

- If upstream freshness metadata exists, it is propagated automatically
- No requirement to retain event-time columns in derived schemas

**Pipelines without temporal signals**

- No freshness metadata is written (explicitly missing, not inferred)

How users access freshness information
- Via Hudi timeline / commit metadata APIs
- Via external tooling or observability systems
- No new SQL functions or query engine changes included in this RFC

### Hudi RFC Requirements

**Non-Goals**

This RFC explicitly does **not**:

- Change table schemas
- Require SQL or planner changes
- Enforce freshness SLAs
- Infer missing timestamps
- Introduce per-record freshness tracking

**Backward Compatibility**

- Commit metadata extension is additive
- No file format or schema changes
- No behavioral changes when feature is disabled

Fully backward compatible.

**Alternatives Considered**

**Table columns for freshness**
Rejected due to schema impact and merge complexity.

**Query-engine computation**
Rejected due to non-determinism and engine coupling.

**External SLA systems**
Rejected due to lack of atomicity and replayability.

**Future Work (Out of Scope)**

- Standardized freshness schema
- Metadata table integration
- Optional SQL exposure
- Multi-table freshness lineage visualization

**Summary**

This proposal adds a **minimal, deterministic, and engine-independent** mechanism for tracking data freshness in Apache Hudi by leveraging commit metadata.

It provides meaningful freshness observability while preserving Hudi’s core design principles: immutability, backward compatibility, and separation of storage and compute concerns.