manifests metadata table reports file counts in both data and delete columns
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
The `manifests` metadata table writes every manifest's added/existing/deleted counts into **both** the `*_data_files_count` and `*_delete_files_count` columns, regardless of the manifest's content type — a data manifest reports phantom delete-file counts, and a delete manifest reports phantom data-file counts.
The reference implementation gates each column by content type (`ManifestsTable.java`, `manifestFileToRow`):
```java
manifest.content() == ManifestContent.DATA ? manifest.addedFilesCount() : 0, // added_data_files_count
manifest.content() == ManifestContent.DELETES ? manifest.addedFilesCount() : 0, // added_delete_files_count
```
In `crates/iceberg/src/inspect/manifests.rs` the six `append_value` sites use the same `ManifestFile` counts unconditionally. Easy to see with a snapshot containing one data and one deletes manifest: both rows show identical counts in all six columns.
Found while exploring #823. I have a fix with tests ready; PR incoming.
Contributor guide
Research direction
Start in crates/iceberg/src/inspect/manifests.rs at the six append_value sites and compare their behavior with the content-type rules shown from ManifestsTable.java. Reproduce with a snapshot containing one data and one deletes manifest, then run the relevant manifest inspection tests; done means each row reports counts only in its matching data or delete columns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 39/100