Found commits after time :20230220161017756, please rollback greater commits first
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
Hi,
Our streaming job is failing with Found commits after time :20230220161017756, please rollback greater commits first,
We tried to rollback commits with the command commit rollback --commit commit_num but we are facing the below exception
Caused by: java.lang.IllegalArgumentException: Cannot use marker based rollback strategy on completed instant:[20230221130513323__deltacommit__COMPLETED]
at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40)
at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.(BaseRollbackActionExecutor.java:93)
at org.apache.hudi.table.action.rollback.BaseRollbackActionExecutor.(BaseRollbackActionExecutor.java:73)
at org.apache.hudi.table.action.rollback.MergeOnReadRollbackActionExecutor.(MergeOnReadRollbackActionExecutor.java:48)
at org.apache.hudi.table.HoodieSparkMergeOnReadTable.rollback(HoodieSparkMergeOnReadTable.java:170)
at org.apache.hudi.client.BaseHoodieWriteClient.rollback(BaseHoodieWriteClient.java:766)
... 15 more
Our Hudi configs are
DataSourceWriteOptions.TABLE_TYPE.key() -> DataSourceWriteOptions.MOR_TABLE_TYPE_OPT_VAL,
DataSourceWriteOptions.RECORDKEY_FIELD.key() -> "guid",
DataSourceWriteOptions.PARTITIONPATH_FIELD.key() -> "collectionName",
DataSourceWriteOptions.PRECOMBINE_FIELD.key() -> "operationTime",
HoodieCompactionConfig.INLINE_COMPACT_TRIGGER_STRATEGY.key() -> CompactionTriggerStrategy.TIME_ELAPSED.name,
HoodieCompactionConfig.INLINE_COMPACT_TIME_DELTA_SECONDS.key() -> String.valueOf(60 * 60),
HoodieCompactionConfig.CLEANER_POLICY.key() -> HoodieCleaningPolicy.KEEP_LATEST_COMMITS.name(),
HoodieCompactionConfig.CLEANER_COMMITS_RETAINED.key() -> "624",
HoodieCompactionConfig.MIN_COMMITS_TO_KEEP.key() -> "625",
HoodieCompactionConfig.MAX_COMMITS_TO_KEEP.key() -> "648",
HoodieCompactionConfig.ASYNC_CLEAN.key() -> "false",
HoodieCompactionConfig.INLINE_COMPACT.key() -> "true",
HoodieMetricsConfig.TURN_METRICS_ON.key() -> "true",
HoodieMetricsConfig.METRICS_REPORTER_TYPE_VALUE.key() -> MetricsReporterType.DATADOG.name(),
HoodieMetricsDatadogConfig.API_SITE_VALUE.key() -> "US",
HoodieMetricsDatadogConfig.METRIC_PREFIX_VALUE.key() -> "tacticalnovusingest.hudi",
HoodieMetricsDatadogConfig.API_KEY_SUPPLIER.key() -> "com.tr.indigo.tacticalnovusingest.utils.DatadogKeySupplier",
HoodieMetadataConfig.ENABLE.key() -> "false",
HoodieWriteConfig.ROLLBACK_USING_MARKERS_ENABLE.key() -> "false",
Please help what caused this issue
Help us how to resolve this
**Environment Description** AWS
* Hudi version : 0.11.1
* Spark version : 3.1.2
* Hive version : NA
* Hadoop version :3
* Storage (HDFS/S3/GCS..) :S3
* Running on Docker? (yes/no) :no
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```Add the stacktrace of the error.```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the rollback path through BaseHoodieWriteClient.rollback, HoodieSparkMergeOnReadTable.rollback, and BaseRollbackActionExecutor, using the reported completed instant and rollback configuration as the reproducer. Check existing rollback tests for completed instants and marker-based strategies. Done means the cause is reproduced and a verified resolution or clearly documented limitation is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java, spark
- Domain
- data-engineering, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100