Timeline layout v2 path configs are persisted into hoodie.properties of tables below version 8
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
A table created below table version 8 gets `hoodie.timeline.path=timeline` and `hoodie.timeline.history.path=history` written into its `hoodie.properties`, even though it does not use the layout those configs describe.
Both configs were introduced with timeline layout version 2 in 1.0.0 and are read only by `TimelinePathProviderV2`, which is selected exclusively at layout version 2 (table version 8 and above). A table below that version keeps its timeline directly under `.hoodie` (`TimelinePathProviderV1` hardcodes it) and resolves its archived timeline through `hoodie.archivelog.folder`. Neither config has any meaning there.
The cause is that `HoodieTableConfig.dropInvalidConfigs()` only strips a config that declares a `sinceVersion`:
```java
if (!configProperty.getSinceVersion().isPresent()) {
return true; // never dropped
}
```
`RECORD_MERGE_MODE` declares `.sinceVersion("1.0.0")` and is correctly dropped from a version 6 table; `TIMELINE_PATH` and `TIMELINE_HISTORY_PATH` declare nothing and survive.
The immediate effect is cosmetic, since every consumer resolves these through `getStringOrDefault`. It matters because table version 6 is the interoperable format used when writing with a 1.x binary for a 0.x reader, and the 0.x line has no definition for either key. Recording storage layout that the table does not have is also a trap for anything that later starts honouring the values.
Reproduce by creating a table with `hoodie.write.table.version=6` and reading `hoodie.properties`.
Fix: declare the introducing version on both so the existing gate drops them below table version 8.
Note that `hoodie.table.format` looks similar but must not be handled this way: the table format SPI is orthogonal to the table version, nothing gates a custom format on table version 9, and no upgrade handler restores the config, so gating it on `sinceVersion` would permanently discard a custom format on any table below version 9.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.