apache / apache/carbondata

SDKS3Example is not working

Open
#4,304 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.5k
Forks
694
PR merge metrics
No merged PRs in 30d

Description

There are some error when I run the example :org.apache.carbondata.examples.sdk.SDKS3Example

Errors:

```
2023-04-09 14:35:16 INFO CarbonProperties:1700 - Considered value for min max byte limit for string is: 200
2023-04-09 14:35:16 INFO CarbonProperties:1725 - Using default value for carbon.detail.batch.size 100
2023-04-09 14:35:17 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2023-04-09 14:35:17 INFO CarbonDataProcessorUtil:100 - Successfully created dir: /var/folders/lw/4y5plg0x7rq45h38m4sfxlbm0000gn/T//08f6e51ef0ba4ffda2c6f3ad15c7baf2_attempt_cfaf6637-f2e9-4b0b-9258-7dd5a4494e51_0000_m_-811962868_1490758150
2023-04-09 14:35:19 INFO DataLoadExecutor:49 - Data Loading is started for table _temptable-c7c3e84c-37d4-4ea3-aaea-12d4b7d53c20_1681022116553
2023-04-09 14:35:19 INFO CarbonDataProcessorUtil:100 - Successfully created dir: /var/folders/lw/4y5plg0x7rq45h38m4sfxlbm0000gn/T//08f6e51ef0ba4ffda2c6f3ad15c7baf2_attempt_cfaf6637-f2e9-4b0b-9258-7dd5a4494e51_0000_m_-811962868_1490758150/Fact/Part0/Segment_null/57cb395e3c304c7ba51bfd6215df27e4
2023-04-09 14:35:21 WARN CarbonOutputIteratorWrapper:87 - try to poll a row batch one more time.
2023-04-09 14:35:21 INFO AbstractFactDataWriter:172 - Total file size: 1073741824 and dataBlock Size: 966367642
2023-04-09 14:35:21 INFO AbstractFactDataWriter:181 - Carbondata will directly write fact data to store path.
2023-04-09 14:35:21 INFO CarbonFactDataWriterImplV3:98 - Sort Scope : NO_SORT
2023-04-09 14:35:21 WARN CarbonOutputIteratorWrapper:87 - try to poll a row batch one more time.
2023-04-09 14:35:21 WARN CarbonOutputIteratorWrapper:87 - try to poll a row batch one more time.
2023-04-09 14:35:21 WARN UnsafeMemoryManager:83 - It is not recommended to set off-heap working memory size less than 512MB, so setting default value to 512
2023-04-09 14:35:21 INFO UnsafeMemoryManager:109 - Off-heap Working Memory manager is created with size 536870912 with OFFHEAP
2023-04-09 14:35:21 INFO CarbonFactDataWriterImplV3:178 - Number of Pages for blocklet is: 1 :Rows Added: 3
2023-04-09 14:38:51 ERROR CarbonFactDataWriterImplV3:390 - Problem while writing the index file
org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
2023-04-09 14:38:51 ERROR CarbonRowDataWriterProcessorStepImpl:279 - Problem while writing the index file
org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
2023-04-09 14:38:51 ERROR CarbonRowDataWriterProcessorStepImpl:160 - Failed for table: _temptable-c7c3e84c-37d4-4ea3-aaea-12d4b7d53c20_1681022116553 in DataWriterProcessorStepImpl
org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:260)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:280)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
... 10 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
... 11 more
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
2023-04-09 14:38:51 INFO AbstractDataLoadProcessorStep:139 - Total rows processed in step Data Writer: 3
2023-04-09 14:38:51 INFO AbstractDataLoadProcessorStep:139 - Total rows processed in step Data Converter: 3
2023-04-09 14:38:51 INFO AbstractDataLoadProcessorStep:139 - Total rows processed in step Input Processor: 3
2023-04-09 14:38:51 ERROR CarbonTableOutputFormat:481 - Error while loading data
java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$CarbonRecordWriter.close(CarbonTableOutputFormat.java:479)
at org.apache.carbondata.sdk.file.CSVCarbonWriter.close(CSVCarbonWriter.java:166)
at org.apache.carbondata.examples.sdk.SDKS3Example.main(SDKS3Example.java:87)
Caused by: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:292)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:162)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
... 6 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:260)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
... 8 more
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:280)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
... 10 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
... 11 more
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$CarbonRecordWriter.close(CarbonTableOutputFormat.java:482)
at org.apache.carbondata.sdk.file.CSVCarbonWriter.close(CSVCarbonWriter.java:166)
at org.apache.carbondata.examples.sdk.SDKS3Example.main(SDKS3Example.java:87)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat$CarbonRecordWriter.close(CarbonTableOutputFormat.java:479)
... 2 more
Caused by: java.lang.RuntimeException: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:292)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException: Error while initializing data handler :
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:162)
at org.apache.carbondata.processing.loading.DataLoadExecutor.execute(DataLoadExecutor.java:51)
at org.apache.carbondata.hadoop.api.CarbonTableOutputFormat.lambda$getRecordWriter$0(CarbonTableOutputFormat.java:280)
... 6 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:260)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.doExecute(CarbonRowDataWriterProcessorStepImpl.java:221)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.execute(CarbonRowDataWriterProcessorStepImpl.java:146)
... 8 more
Caused by: org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException:
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:280)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.finish(CarbonRowDataWriterProcessorStepImpl.java:255)
... 10 more
Caused by: org.apache.carbondata.core.datastore.exception.CarbonDataWriterException: Problem while writing the index file
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:391)
at org.apache.carbondata.processing.store.CarbonFactDataHandlerColumnar.closeHandler(CarbonFactDataHandlerColumnar.java:500)
at org.apache.carbondata.processing.loading.steps.CarbonRowDataWriterProcessorStepImpl.processingComplete(CarbonRowDataWriterProcessorStepImpl.java:277)
... 11 more
Caused by: org.apache.carbondata.core.exception.CarbonFileException: Unable to get file status:
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:115)
at org.apache.carbondata.core.datastore.filesystem.S3CarbonFile.getDataOutputStream(S3CarbonFile.java:100)
at org.apache.carbondata.core.datastore.impl.FileFactory.getDataOutputStream(FileFactory.java:230)
at org.apache.carbondata.core.writer.ThriftWriter.open(ThriftWriter.java:82)
at org.apache.carbondata.core.writer.CarbonIndexFileWriter.openThriftWriter(CarbonIndexFileWriter.java:53)
at org.apache.carbondata.processing.store.writer.AbstractFactDataWriter.writeIndexFile(AbstractFactDataWriter.java:454)
at org.apache.carbondata.processing.store.writer.v3.CarbonFactDataWriterImplV3.closeWriter(CarbonFactDataWriterImplV3.java:388)
... 13 more
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://obs-xubo4/sdk/57cb395e3c304c7ba51bfd6215df27e4_batchno0-0-null-1681022116553.carbonindex
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:996)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:77)
at org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getAbsolutePath(AbstractDFSCarbonFile.java:113)
... 19 more
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Run org.apache.carbondata.examples.sdk.SDKS3Example with the reported S3A path and compare the failure with S3CarbonFile.java and AbstractDFSCarbonFile.java, following the stack trace from CarbonFactDataWriterImplV3. Done means the example completes successfully and writes the carbonindex file to the configured s3a location without the reported FileNotFoundException.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, java, scala
Domain
cloud, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.