apache / apache/hudi

[SUPPORT] Is it allowed using Flink Table API sqlQuery() to read data from hudi tables?

Open
#9,093 7 comments 0 reactions 0 assignees View on GitHub
engine:flink type:feature
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**Describe the problem you faced**

I'm trying to use flink table api sqlQuery to read data from hudi table but not working, so am i doing it wrong or hudi doesn't support this way to query data.

**Code**
```java
sEnv = StreamExecutionEnvironment.getExecutionEnvironment();
sTableEnv = StreamTableEnvironment.create(sEnv);
sEnv.setParallelism(1);
sEnv.enableCheckpointing(3000);
// create table
String createTabelSql = "create table dept(\n" +
" dept_id BIGINT PRIMARY KEY NOT ENFORCED,\n" +
" dept_name varchar(10),\n" +
" ts timestamp(3)\n" +
")\n" +
"with (\n" +
" 'connector' = 'hudi',\n" +
" 'path' = 'hdfs://localhost:9000/hudi/dept',\n" +
" 'table.type' = 'MERGE_ON_READ'\n" +
")";
sTableEnv.executeSql(createTabelSql);
// insert data
sTableEnv.executeSql("insert into dept values (1, 'a', NOW()), (2, 'b', NOW())");
// query data
Table table = sTableEnv.sqlQuery("select * from dept");
DataStream dataStream = sTableEnv.toDataStream(table);
// there's nothing to print
dataStream.print();
```

**Environment Description**

* Hudi version : 1.12.0

* Hadoop version : 3.1.3

* Flink version: 1.13.6

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the shown Flink Table API flow: create the Hudi table, insert the sample rows, call sqlQuery("select * from dept"), and convert it with toDataStream. Compare the observed behavior with the Hudi and Flink integration documentation; done means determining whether this query path is supported and documenting or correcting the confirmed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.