delta-io / delta-io/delta-kernel-rs

Seed file stats from single-file checkpoints (classic and V2)

Open
#2,011 0 comments 0 reactions 1 assignee Claimed by @scottsand-db View on GitHub
Dominant language
Rust
Stars
361
Forks
214
Avg merge
2d 17h
Merged PRs (30d)
87

Description

When seeding CRC context from a classic or V2 single-file checkpoint, we already scan the entire file to extract metadata fields. Since add/remove actions are in the same file, we should also extract file counts and sizes to produce `Valid` file stats instead of `Incomplete`.

This only applies to single-file checkpoints -- multi-part and V2 manifest+sidecar checkpoints have add/remove actions spread across multiple files, so the cost trade-off is different.

Follow-up to #1781.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.