[SUPPORT] Archiving the commits failing for Hudi job

**_Tips before filing an issue_**

- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? Yes

- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.

**Describe the problem you faced**

Hi Team, We have data pipeline that writes data into Hudi tables after reading from sources. After few days of successful runs, job started failing while archiving the commit timelines.

**To Reproduce**:

We couldn't produce the error, as it happened only once and then all the subsequent runs failed after that.

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment Description** : Apache Hudi on EMR cluster

* Hudi version : 2.12-0.7.0-amzn-0.jar

* Spark version : 3.1.1-amzn-0.1

* Hive version :

* Hadoop version :

* Storage (HDFS/S3/GCS..) : S3

* Running on Docker? (yes/no) : No

**Additional context**

Add any other context about the problem here.

**Stacktrace**

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

22/10/29 11:05:34 INFO HoodieLogFormat$WriterBuilder: Computed the next log version for commits in s3://bucket/dqip_data/.hoodie/archived as 35 with write-token 1-0-1
22/10/29 11:05:34 INFO HoodieLogFormat$WriterBuilder: HoodieLogFile on path s3://bucket/dqip_data/.hoodie/archived/.commits_.archive.35_1-0-1
22/10/29 11:05:34 INFO HoodieTimelineArchiveLog: Archiving instants [[==>20221028122716__replacecommit__REQUESTED], [==>20221028122716__replacecommit__INFLIGHT], [20221028122716__replacecommit__COMPLETED], [==>20221028141111__replacecommit__REQUESTED], [==>20221028141111__replacecommit__INFLIGHT], [20221028141111__replacecommit__COMPLETED], [==>20221028141409__replacecommit__REQUESTED], [==>20221028141409__replacecommit__INFLIGHT], [20221028141409__replacecommit__COMPLETED], [==>20221028142716__replacecommit__REQUESTED], [==>20221028142716__replacecommit__INFLIGHT], [20221028142716__replacecommit__COMPLETED], [==>20221028161416__replacecommit__REQUESTED], [==>20221028161416__replacecommit__INFLIGHT], [20221028161416__replacecommit__COMPLETED], [==>20221028162846__replacecommit__REQUESTED], [==>20221028162846__replacecommit__INFLIGHT], [20221028162846__replacecommit__COMPLETED], [==>20221028165845__replacecommit__REQUESTED], [==>20221028165845__replacecommit__INFLIGHT], [20221028165845__replacecommit__COMPLETED], [==>20221028181416__replacecommit__REQUESTED], [==>20221028181416__replacecommit__INFLIGHT], [20221028181416__replacecommit__COMPLETED], [==>20221028182709__replacecommit__REQUESTED], [==>20221028182709__replacecommit__INFLIGHT], [20221028182709__replacecommit__COMPLETED], [==>20221028201423__replacecommit__REQUESTED], [==>20221028201423__replacecommit__INFLIGHT], [20221028201423__replacecommit__COMPLETED], [==>20221028203806__replacecommit__REQUESTED], [==>20221028203806__replacecommit__INFLIGHT], [20221028203806__replacecommit__COMPLETED]]
22/10/29 11:05:34 INFO HoodieTimelineArchiveLog: Wrapper schema {"type":"record","name":"HoodieArchivedMetaEntry","namespace":"org.apache.hudi.avro.model","fields":[{"name":"hoodieCommitMetadata","type":["null",{"type":"record","name":"HoodieCommitMetadata","fields":[{"name":"partitionToWriteStats","type":["null",{"type":"map","values":{"type":"array","items":{"type":"record","name":"HoodieWriteStat","fields":[{"name":"fileId","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"path","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"prevCommit","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"numWrites","type":["null","long"],"default":null},{"name":"numDeletes","type":["null","long"],"default":null},{"name":"numUpdateWrites","type":["null","long"],"default":null},{"name":"totalWriteBytes","type":["null","long"],"default":null},{"name":"totalWriteErrors","type":["null","long"],"default":null},{"name":"partitionPath","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"totalLogRecords","type":["null","long"],"default":null},{"name":"totalLogFiles","type":["null","long"],"default":null},{"name":"totalUpdatedRecordsCompacted","type":["null","long"],"default":null},{"name":"numInserts","type":["null","long"],"default":null},{"name":"totalLogBlocks","type":["null","long"],"default":null},{"name":"totalCorruptLogBlock","type":["null","long"],"default":null},{"name":"totalRollbackBlocks","type":["null","long"],"default":null},{"name":"fileSizeInBytes","type":["null","long"],"default":null}]}},"avro.java.string":"String"}],"default":null},{"name":"extraMetadata","type":["null",{"type":"map","values":{"type":"string","avro.java.string":"String"},"avro.java.string":"String"}],"default":null},{"name":"version","type":["int","null"],"default":1},{"name":"operationType","type":["null",{"type":"string","avro.java.string":"String"}],"default":null}]}],"default":null},{"name":"hoodieCleanMetadata","type":["null",{"type":"record","name":"HoodieCleanMetadata","fields":[{"name":"startCleanTime","type":{"type":"string","avro.java.string":"String"}},{"name":"timeTakenInMillis","type":"long"},{"name":"totalFilesDeleted","type":"int"},{"name":"earliestCommitToRetain","type":{"type":"string","avro.java.string":"String"}},{"name":"partitionMetadata","type":{"type":"map","values":{"type":"record","name":"HoodieCleanPartitionMetadata","fields":[{"name":"partitionPath","type":{"type":"string","avro.java.string":"String"}},{"name":"policy","type":{"type":"string","avro.java.string":"String"}},{"name":"deletePathPatterns","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"successDeleteFiles","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"failedDeleteFiles","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}}]},"avro.java.string":"String"}},{"name":"version","type":["int","null"],"default":1},{"name":"bootstrapPartitionMetadata","type":["null",{"type":"map","values":"HoodieCleanPartitionMetadata","avro.java.string":"String","default":null}],"default":null}]}],"default":null},{"name":"hoodieCompactionMetadata","type":["null",{"type":"record","name":"HoodieCompactionMetadata","fields":[{"name":"partitionToCompactionWriteStats","type":["null",{"type":"map","values":{"type":"array","items":{"type":"record","name":"HoodieCompactionWriteStat","fields":[{"name":"partitionPath","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"totalLogRecords","type":["null","long"],"default":null},{"name":"totalLogFiles","type":["null","long"],"default":null},{"name":"totalUpdatedRecordsCompacted","type":["null","long"],"default":null},{"name":"hoodieWriteStat","type":["null","HoodieWriteStat"],"default":null}]}},"avro.java.string":"String"}]}]}],"default":null},{"name":"hoodieRollbackMetadata","type":["null",{"type":"record","name":"HoodieRollbackMetadata","fields":[{"name":"startRollbackTime","type":{"type":"string","avro.java.string":"String"}},{"name":"timeTakenInMillis","type":"long"},{"name":"totalFilesDeleted","type":"int"},{"name":"commitsRollback","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"partitionMetadata","type":{"type":"map","values":{"type":"record","name":"HoodieRollbackPartitionMetadata","fields":[{"name":"partitionPath","type":{"type":"string","avro.java.string":"String"}},{"name":"successDeleteFiles","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"failedDeleteFiles","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}},{"name":"rollbackLogFiles","type":["null",{"type":"map","values":"long","avro.java.string":"String"}],"default":null},{"name":"writtenLogFiles","type":["null",{"type":"map","values":"long","avro.java.string":"String"}],"default":null}]},"avro.java.string":"String"}},{"name":"version","type":["int","null"],"default":1},{"name":"instantsRollback","type":{"type":"array","items":{"type":"record","name":"HoodieInstantInfo","fields":[{"name":"commitTime","type":{"type":"string","avro.java.string":"String"}},{"name":"action","type":{"type":"string","avro.java.string":"String"}}]},"default":null},"default":null}]}],"default":null},{"name":"hoodieSavePointMetadata","type":["null",{"type":"record","name":"HoodieSavepointMetadata","fields":[{"name":"savepointedBy","type":{"type":"string","avro.java.string":"String"}},{"name":"savepointedAt","type":"long"},{"name":"comments","type":{"type":"string","avro.java.string":"String"}},{"name":"partitionMetadata","type":{"type":"map","values":{"type":"record","name":"HoodieSavepointPartitionMetadata","fields":[{"name":"partitionPath","type":{"type":"string","avro.java.string":"String"}},{"name":"savepointDataFile","type":{"type":"array","items":{"type":"string","avro.java.string":"String"}}}]},"avro.java.string":"String"}},{"name":"version","type":["int","null"],"default":1}]}],"default":null},{"name":"commitTime","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"actionType","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"version","type":["int","null"],"default":1},{"name":"hoodieCompactionPlan","type":["null",{"type":"record","name":"HoodieCompactionPlan","fields":[{"name":"operations","type":["null",{"type":"array","items":{"type":"record","name":"HoodieCompactionOperation","fields":[{"name":"baseInstantTime","type":["null",{"type":"string","avro.java.string":"String"}]},{"name":"deltaFilePaths","type":["null",{"type":"array","items":{"type":"string","avro.java.string":"String"}}],"default":null},{"name":"dataFilePath","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"fileId","type":["null",{"type":"string","avro.java.string":"String"}]},{"name":"partitionPath","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"metrics","type":["null",{"type":"map","values":"double","avro.java.string":"String"}],"default":null},{"name":"bootstrapFilePath","type":["null",{"type":"string","avro.java.string":"String"}],"default":null}]}}],"default":null},{"name":"extraMetadata","type":["null",{"type":"map","values":{"type":"string","avro.java.string":"String"},"avro.java.string":"String"}],"default":null},{"name":"version","type":["int","null"],"default":1}]}],"default":null},{"name":"hoodieCleanerPlan","type":["null",{"type":"record","name":"HoodieCleanerPlan","fields":[{"name":"earliestInstantToRetain","type":["null",{"type":"record","name":"HoodieActionInstant","fields":[{"name":"timestamp","type":{"type":"string","avro.java.string":"String"}},{"name":"action","type":{"type":"string","avro.java.string":"String"}},{"name":"state","type":{"type":"string","avro.java.string":"String"}}]}],"default":null},{"name":"policy","type":{"type":"string","avro.java.string":"String"}},{"name":"filesToBeDeletedPerPartition","type":["null",{"type":"map","values":{"type":"array","items":{"type":"string","avro.java.string":"String"}},"avro.java.string":"String"}],"default":null},{"name":"version","type":["int","null"],"default":1},{"name":"filePathsToBeDeletedPerPartition","type":["null",{"type":"map","values":{"type":"array","items":{"type":"record","name":"HoodieCleanFileInfo","fields":[{"name":"filePath","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"isBootstrapBaseFile","type":["null","boolean"],"default":null}]}},"avro.java.string":"String"}],"doc":"This field replaces the field filesToBeDeletedPerPartition","default":null}]}],"default":null},{"name":"actionState","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"hoodieReplaceCommitMetadata","type":["null",{"type":"record","name":"HoodieReplaceCommitMetadata","fields":[{"name":"partitionToWriteStats","type":["null",{"type":"map","values":{"type":"array","items":"HoodieWriteStat"},"avro.java.string":"String"}],"default":null},{"name":"extraMetadata","type":["null",{"type":"map","values":{"type":"string","avro.java.string":"String"},"avro.java.string":"String"}],"default":null},{"name":"version","type":["int","null"],"default":1},{"name":"operationType","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"partitionToReplaceFileIds","type":["null",{"type":"map","values":{"type":"array","items":{"type":"string","avro.java.string":"String"}},"avro.java.string":"String"}],"default":null}]}],"default":null}]}
22/10/29 11:05:34 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:35 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:36 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 INFO AbstractTableFileSystemView: Took 2568 ms to read 31 instants, 36565 replaced file groups
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:37 WARN ClusteringUtils: No content found in requested file for instant [==>20221029110511__replacecommit__INFLIGHT]
22/10/29 11:05:37 INFO ClusteringUtils: Found 0 files in pending clustering operations
22/10/29 11:05:37 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:38 WARN DAGScheduler: Broadcasting large task binary with size 2.2 MiB
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:40 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:41 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:42 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 INFO AbstractTableFileSystemView: Took 3338 ms to read 31 instants, 36565 replaced file groups
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:43 WARN ClusteringUtils: No content found in requested file for instant [==>20221029110511__replacecommit__INFLIGHT]
22/10/29 11:05:43 INFO ClusteringUtils: Found 0 files in pending clustering operations
22/10/29 11:05:43 WARN HadoopFileSystemOwner: found no group information for livy (auth:SIMPLE), using livy as primary group
22/10/29 11:05:44 ERROR default: ****************An error occurred while calling o177.save.
: org.apache.hudi.exception.HoodieCommitException: Failed to archive commits
at org.apache.hudi.table.HoodieTimelineArchiveLog.archive(HoodieTimelineArchiveLog.java:322)
at org.apache.hudi.table.HoodieTimelineArchiveLog.archiveIfRequired(HoodieTimelineArchiveLog.java:138)
at org.apache.hudi.client.AbstractHoodieWriteClient.postCommit(AbstractHoodieWriteClient.java:426)
at org.apache.hudi.client.AbstractHoodieWriteClient.commitStats(AbstractHoodieWriteClient.java:188)
at org.apache.hudi.client.SparkRDDWriteClient.commit(SparkRDDWriteClient.java:110)
at org.apache.hudi.HoodieSparkSqlWriter$.commitAndPerformPostOperations(HoodieSparkSqlWriter.scala:442)
at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:218)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:134)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:90)
at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:185)
at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:223)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:220)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:181)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:134)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:133)
at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:989)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)
at org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:232)
at org.apache.spark.sql.execution.SQLExecution$.executeQuery$1(SQLExecution.scala:110)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:135)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)
at org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:232)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:135)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:253)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:134)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:772)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:68)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:989)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:438)
at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:415)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:293)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: Positive number of partitions required
at org.apache.spark.rdd.ParallelCollectionRDD$.slice(ParallelCollectionRDD.scala:118)
at org.apache.spark.rdd.ParallelCollectionRDD.getPartitions(ParallelCollectionRDD.scala:96)
at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:296)
at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.rdd.RDD.partitions(RDD.scala:296)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2303)
at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1030)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)
at org.apache.spark.rdd.RDD.collect(RDD.scala:1029)
at org.apache.spark.api.java.JavaRDDLike.collect(JavaRDDLike.scala:362)
at org.apache.spark.api.java.JavaRDDLike.collect$(JavaRDDLike.scala:361)
at org.apache.spark.api.java.AbstractJavaRDDLike.collect(JavaRDDLike.scala:45)
at org.apache.hudi.client.common.HoodieSparkEngineContext.map(HoodieSparkEngineContext.java:73)
at org.apache.hudi.client.ReplaceArchivalHelper.deleteReplacedFileGroups(ReplaceArchivalHelper.java:72)
at org.apache.hudi.table.HoodieTimelineArchiveLog.deleteReplacedFileGroups(HoodieTimelineArchiveLog.java:341)
at org.apache.hudi.table.HoodieTimelineArchiveLog.archive(HoodieTimelineArchiveLog.java:303)
... 45 more
********************
22/10/29 11:05:44 ERROR ApplicationMaster: User application exited with status 1

Start by reviewing the supplied archive log and environment details for the EMR/Spark job writing to S3. The report provides no reproducible case, exception, source file, or test; a complete stacktrace and a reliable reproduction would be needed to define what done looks like.

Written by the indexing model from the issue text.