hudi-cli: export instants fails on v8+ tables, ignores --limit, and has no restore case
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Bug Description
**What happened:**
`export instants` in hudi-cli has three defects on current tables (master 6ac27904, `ExportCommand.java`):
1. On any table of version 8 or above, the default (ascending) order fails before exporting anything, even with zero archived instants. Line 102 calls `globEntries(archivePath)` on `.hoodie/timeline/history`; with no wildcard the glob returns the directory itself, and `copyArchivedInstants` (line 124) opens it as a log file:
`HoodieIOException: Exception creating input stream from file: .../.hoodie/timeline/history`.
Beyond that, `copyArchivedInstants` reads `HoodieArchivedMetaEntry` records from the pre-v8 log format, while the v8+ history is the LSM timeline (parquet), so the archived branch cannot work on current tables at all.
2. `--limit` is ignored for active instants: `copyNonArchivedInstants(instants, limit, localFolder)` (line 190) never reads its `limit` parameter. `export instants --limit 2 --desc true` on a table with 3 commits prints `Exported 3 Instants`.
3. The default `--actions` list (line 80) includes `restore`, but the switch in `copyNonArchivedInstants` has no `restore` case, so a completed restore instant on the timeline makes the export throw `HoodieException("Unknown type of action restore")` (line 228).
**What you expected:**
Exporting a v8+ table works in either order, honours `--limit`, and handles every action in its own default filter. The archived branch should read the LSM history through `ArchivedTimelineV2` / `LSMTimeline`, the way #18816 reworked `ArchivedCommitsCommand`.
**Steps to reproduce:**
1. Create a table on master, write three commits.
2. `connect --path ` then `export instants --localFolder /tmp/out` -> HoodieIOException on the history directory.
3. `export instants --localFolder /tmp/out --desc true --limit 2` -> 3 files exported.
Found while writing `TestExportCommand` in #19877, which works around the first two by always passing `--desc true` with a limit equal to the instant count.
### Environment
**Hudi version:** master (6ac27904); table version 8 and 9
**Query engine:** hudi-cli (Spark)
**Relevant configs:** none
### Logs and Stack Trace
```
org.apache.hudi.exception.HoodieIOException: Exception creating input stream from file: file:/.../.hoodie/timeline/history
at org.apache.hudi.cli.commands.ExportCommand.copyArchivedInstants(ExportCommand.java)
at org.apache.hudi.cli.commands.ExportCommand.exportInstants(ExportCommand.java:115)
```
### Related
- Origin: #8608 (2023) replaced the `.commits_.archive*` glob with the bare archive path, which is what now resolves to the history directory, and removed the `++copyCount == limit` check from `copyNonArchivedInstants`, leaving `limit` unused.
- The Spark procedure `export_instants` (`ExportInstantsProcedure.scala`) is a port of this command and still reads archived instants as `HoodieArchivedMetaEntry` log records, so its archived branch is equally dead on version 8+ tables. #19172 fixed only its descending order.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with hudi-cli's ExportCommand.java and TestExportCommand from #19877, then compare the archived-instant handling with ArchivedCommitsCommand and its ArchivedTimelineV2/LSMTimeline use in #18816. Also inspect ExportInstantsProcedure.scala because it shares the archived-path behavior. Done means v8+ exports work in both orders, --limit is honored, and restore actions pass the command's tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala, spark
- Domain
- cli, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100