apache / apache/hudi

hudi-cli: export instants fails on v8+ tables, ignores --limit, and has no restore case

Open
#19,879 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

### Bug Description

**What happened:**
`export instants` in hudi-cli has three defects on current tables (master 6ac27904, `ExportCommand.java`):

1. On any table of version 8 or above, the default (ascending) order fails before exporting anything, even with zero archived instants. Line 102 calls `globEntries(archivePath)` on `.hoodie/timeline/history`; with no wildcard the glob returns the directory itself, and `copyArchivedInstants` (line 124) opens it as a log file:
`HoodieIOException: Exception creating input stream from file: .../.hoodie/timeline/history`.
Beyond that, `copyArchivedInstants` reads `HoodieArchivedMetaEntry` records from the pre-v8 log format, while the v8+ history is the LSM timeline (parquet), so the archived branch cannot work on current tables at all.
2. `--limit` is ignored for active instants: `copyNonArchivedInstants(instants, limit, localFolder)` (line 190) never reads its `limit` parameter. `export instants --limit 2 --desc true` on a table with 3 commits prints `Exported 3 Instants`.
3. The default `--actions` list (line 80) includes `restore`, but the switch in `copyNonArchivedInstants` has no `restore` case, so a completed restore instant on the timeline makes the export throw `HoodieException("Unknown type of action restore")` (line 228).

**What you expected:**
Exporting a v8+ table works in either order, honours `--limit`, and handles every action in its own default filter. The archived branch should read the LSM history through `ArchivedTimelineV2` / `LSMTimeline`, the way #18816 reworked `ArchivedCommitsCommand`.

**Steps to reproduce:**
1. Create a table on master, write three commits.
2. `connect --path ` then `export instants --localFolder /tmp/out` -> HoodieIOException on the history directory.
3. `export instants --localFolder /tmp/out --desc true --limit 2` -> 3 files exported.

Found while writing `TestExportCommand` in #19877, which works around the first two by always passing `--desc true` with a limit equal to the instant count.

### Environment

**Hudi version:** master (6ac27904); table version 8 and 9
**Query engine:** hudi-cli (Spark)
**Relevant configs:** none

### Logs and Stack Trace

```
org.apache.hudi.exception.HoodieIOException: Exception creating input stream from file: file:/.../.hoodie/timeline/history
at org.apache.hudi.cli.commands.ExportCommand.copyArchivedInstants(ExportCommand.java)
at org.apache.hudi.cli.commands.ExportCommand.exportInstants(ExportCommand.java:115)
```

### Related

- Origin: #8608 (2023) replaced the `.commits_.archive*` glob with the bare archive path, which is what now resolves to the history directory, and removed the `++copyCount == limit` check from `copyNonArchivedInstants`, leaving `limit` unused.
- The Spark procedure `export_instants` (`ExportInstantsProcedure.scala`) is a port of this command and still reads archived instants as `HoodieArchivedMetaEntry` log records, so its archived branch is equally dead on version 8+ tables. #19172 fixed only its descending order.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with hudi-cli's ExportCommand.java and TestExportCommand from #19877, then compare the archived-instant handling with ArchivedCommitsCommand and its ArchivedTimelineV2/LSMTimeline use in #18816. Also inspect ExportInstantsProcedure.scala because it shares the archived-path behavior. Done means v8+ exports work in both orders, --limit is honored, and restore actions pass the command's tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, scala, spark
Domain
cli, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.