[Bug] java api can not read data
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Paimon version
paimon-bundle-0.6
### Compute Engine
java api
### Minimal reproduce step
Here is th official website example:
Identifier identifier = Identifier.create("default", "test");
Options options = new Options();
options.set("warehouse", "hdfs://hadoop100:8020/paimon/hive");
options.set("metastore", "hive");
options.set("uri", "thrift://hadoop100:9083");
options.set("hive-conf-dir", "E:\\tmp\\debzium_test\\src\\main\\resources");
CatalogContext context = CatalogContext.create(options);
Catalog catalog = CatalogFactory.createCatalog(context);
int[] projection = new int[] {0, 1};
Table table = catalog.getTable(identifier);
ReadBuilder readBuilder = table.newReadBuilder();
//.withFilter(Lists.newArrayList(notNull, predicate));
List splits = readBuilder.newScan().plan().splits();
TableRead read = readBuilder.newRead();
RecordReader reader = read.createReader(splits);
reader.forEachRemaining(System.out::println);
### What doesn't meet your expectations?
Here is my example code:
Options options = new Options();
options.set("warehouse", "hdfs://hadoop100:8020/paimon/hive");
options.set("metastore", "hive");
options.set("uri", "thrift://hadoop100:9083");
options.set("hive-conf-dir", "E:\\tmp\\debzium_test\\src\\main\\resources");
CatalogContext context = CatalogContext.create(options);
Catalog catalog = CatalogFactory.createCatalog(context);
Identifier identifier = Identifier.create("default", "test");
Table table = catalog.getTable(identifier);
ReadBuilder readBuilder = table.newReadBuilder();
//.withFilter(Lists.newArrayList(notNull, predicate));
List splits = readBuilder.newScan().plan().splits();
TableRead read = readBuilder.newRead();
RecordReader reader = read.createReader(splits);
//reader.forEachRemaining(row -> System.out.println(row));
RecordReader.RecordIterator iterator = reader.readBatch();
InternalRow next = iterator.next();
long userId = next.getLong(0);
long itemId = next.getLong(1);
BinaryString behavior = next.getString(2);
BinaryString dt = next.getString(3);
BinaryString hh = next.getString(4);
System.out.println("user_id:" + userId + " item_id:" + itemId + " behavior:" + behavior + " dt:" + dt + " hh:" + hh);
### Anything else?
no
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the issue with the Java Catalog, Table, ReadBuilder, TableRead, and RecordReader example in the report, capturing the exception or incorrect result that is currently missing. Compare the working website example with the custom readBatch and iterator path; done means the cause is identified and Java API reads return the expected table data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100