Using UUIDv7 for simultaneous point and time-range pruning in Iceberg
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Feature Request / Improvement
Optimizing an Iceberg table for both time-range queries and ID lookups normally takes two partition fields: a time transform on a `timestamp` column and a bucket transform on an `id` column, usually a UUID. Given only an `id`, Iceberg can pick the bucket but not the time partition, so it has to read that bucket across every partition in the table.
UUIDv7 makes this avoidable, because the identifier already carries its generation time as a Unix timestamp.
The proposal is to give UUIDv7 special handling: keep normal UUID semantics, but expose the embedded timestamp to Iceberg's time transforms. A table partitioned by `day(event_id)` would then prune to a single day for an ID lookup and to a range of days for a time-window query, all from one column.
The goal isn't OLTP 😄. It's to let Iceberg use the time information already present in UUIDv7 for partition and file pruning, from a single column.
_I'm new to Iceberg, so I wouldn't know where to start on this – it just seemed like a good idea while working through some challenges at work, assuming it's feasible._
### Query engine
None
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
No files, tests, or entry points are named. Start by locating Iceberg's UUID handling and partition-transform paths, then determine how UUIDv7 timestamps could support both ID and time-range pruning. Done means the proposal has an agreed design, implementation scope, and tests demonstrating the intended partition and file pruning behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100