[REST Scan Planning] Missing sequence number on `DataFile` object
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 129
Description
### Apache Iceberg version
None
### Query engine
None
### Please describe the bug 🐞
The `DataFile` object, returned by the scan-planning API is missing `sequence-number`, making it impossible to honor `_last_updated_sequence_number` queries with the scan-planning API.
```yaml
DataFile:
allOf:
- $ref: '#/components/schemas/ContentFile'
type: object
required:
- content
properties:
content:
type: string
const: "data"
first-row-id:
type: integer
format: int64
description: "The first row ID assigned to the first row in the data file"
column-sizes:
allOf:
- $ref: '#/components/schemas/CountMap'
description: "Map of column id to total count, including null and NaN"
value-counts:
allOf:
- $ref: '#/components/schemas/CountMap'
description: "Map of column id to null value count"
null-value-counts:
allOf:
- $ref: '#/components/schemas/CountMap'
description: "Map of column id to null value count"
nan-value-counts:
allOf:
- $ref: '#/components/schemas/CountMap'
description: "Map of column id to number of NaN values in the column"
lower-bounds:
allOf:
- $ref: '#/components/schemas/ValueMap'
description: "Map of column id to lower bound primitive type values"
upper-bounds:
allOf:
- $ref: '#/components/schemas/ValueMap'
description: "Map of column id to upper bound primitive type values"
```
Would need an optional:
```yaml
sequence-number:
type: integer
format: int64
description: "The sequence number assigned to the manifest entry or manifest file that this data file belongs to"
```
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
Contributor guide
Research direction
Start by locating the REST scan-planning API schema definition for the DataFile object shown in the issue. Add the optional sequence-number property with the specified type and description, then inspect the scan-planning response path to confirm the value is exposed for DataFile entries. Done means clients can use the returned sequence number to honor _last_updated_sequence_number queries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100