SDKS3Example is not working
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 694
- PR merge metrics
- No merged PRs in 30d
Description
There are some error when I run the example :org.apache.carbondata.examples.sdk.SDKS3Example
Errors:
```
2023-04-09 14:35:16 INFO CarbonProperties:1700 - Considered value for min max byte limit for string is: 200
2023-04-09 14:35:16 INFO CarbonProperties:1725 - Using default value for carbon.detail.batch.size 100
2023-04-09 14:35:17 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2023-04-09 14:35:17 INFO CarbonDataProcessorUtil:100 - Successfully created dir: /var/folders/lw/4y5plg0x7rq45h38m4sfxlbm0000gn/T//08f6e51ef0ba4ffda2c6f3ad15c7baf2_attempt_cfaf6637-f2e9-4b0b-9258-7dd5a4494e51_0000_m_-811962868_1490758150
2023-04-09 14:35:19 INFO DataLoadExecutor:49 - Data Loading is started for table _temptable-c7c3e84c-37d4-4ea3-aaea-12d4b7d53c20_1681022116553
2023-04-09 14:35:19 INFO CarbonDataProcessorUtil:100 - Successfully created dir: /var/folders/lw/4y5plg0x7rq45h38m4sfxlbm0000gn/T//08f6e51ef0ba4ffda2c6f3ad15c7baf2_attempt_cfaf6637-f2e9-4b0b-9258-7dd5a4494e51_0000_m_-811962868_1490758150/Fact/Part0/Segment_null/57cb395e3c304c7ba51bfd6215df27e4
2023-04-09 14:35:21 WARN CarbonOutputIteratorWrapper:87 - try to poll a row batch one more time.
2023-04-09 14:35:21 INFO AbstractFactDataWriter:172 - Total file size: 1073741824 and dataBlock Size: 966367642
2023-04-09 14:35:21 INFO AbstractFactDataWriter:181 - Carbondata will directly write fact data to store path.
2023-04-09 14:35:21 INFO CarbonFactDataWriterImplV3:98 - Sort Scope : NO_SORT
2023-04-09 14:35:21 WARN CarbonOutputIteratorWrapper:87 - try to poll a row batch one more time.
2023-04-09 14:35:21 WARN CarbonOutputIteratorWrapper:87 - try to poll a row batch one more time.
2023-04-09 14:35:21 WARN UnsafeMemoryManager:83 - It is not recommended to set off-heap working memory size less than 512MB, so setting default value to 512
2023-04-09 14:35:21 INFO UnsafeMemoryManager:109 - Off-heap Working Memory manager is created with size 536870912 with OFFHEAP
2023-04-09 14:35:21 INFO CarbonFactDataWriterImplV3:178 - Number of Pages for blocklet is: 1 :Rows Added: 3
2023-04-09 14:38:51 ERROR CarbonFactDataWriterImplV3:390 - Problem while writing the index file
org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
2023-04-09 14:38:51 ERROR CarbonRowDataWriterProcessorStepImpl:279 - Problem while writing the index file
org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
2023-04-09 14:38:51 ERROR CarbonRowDataWriterProcessorStepImpl:160 - Failed for table: _temptable-c7c3e84c-37d4-4ea3-aaea-12d4b7d53c20_1681022116553 in DataWriterProcessorStepImpl
org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:260)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:280)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
... 10 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
... 11 more
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
2023-04-09 14:38:51 INFO AbstractDataLoadProcessorStep:139 - Total rows processed in step Data Writer: 3
2023-04-09 14:38:51 INFO AbstractDataLoadProcessorStep:139 - Total rows processed in step Data Converter: 3
2023-04-09 14:38:51 INFO AbstractDataLoadProcessorStep:139 - Total rows processed in step Input Processor: 3
2023-04-09 14:38:51 ERROR CarbonTableOutputFormat:481 - Error while loading data
java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$CarbonRecordWriter.close(CarbonTableOutputFormat.java:479)
at org.apache.carbondata.sdk.file.CSVCarbonWriter.close(CSVCarbonWriter.java:166)
at org.apache.carbondata.examples.sdk.SDKS3Example.main(SDKS3Example.java:87)
Caused by: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:292)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:162)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
... 6 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:260)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
... 8 more
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:280)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
... 10 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
... 11 more
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$CarbonRecordWriter.close(CarbonTableOutputFormat.java:482)
at org.apache.carbondata.sdk.file.CSVCarbonWriter.close(CSVCarbonWriter.java:166)
at org.apache.carbondata.examples.sdk.SDKS3Example.main(SDKS3Example.java:87)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$CarbonRecordWriter.close(CarbonTableOutputFormat.java:479)
... 2 more
Caused by: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:292)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:162)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
... 6 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:260)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
... 8 more
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:280)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
... 10 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
... 11 more
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Run org.apache.carbondata.examples.sdk.SDKS3Example with the reported S3A path and compare the failure with S3CarbonFile.java and AbstractDFSCarbonFile.java, following the stack trace from CarbonFactDataWriterImplV3. Done means the example completes successfully and writes the carbonindex file to the configured s3a location without the reported FileNotFoundException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java, scala
- Domain
- cloud, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100