apache / apache/beam

[SQL][Java] Fluid query builder

Open
#18,648 0 comments 0 reactions 0 assignees View on GitHub
dsl new feature P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Create a fluid query builder.

 

*Problem*:

SQL queries as java strings are very hard to read and debug. See examples below.

 

*Solution*:

Fluid query builder, along the lines of:
```

QueryTransform query =
QueryBuilder
.from(pcollection)
.where(e -> e.id == 3)
.groupBy(e
-> e.name)
.select();
```

 

*Examples:*
```

String sql = "select f_int2, count(*) as getFieldCount, "
+ "sum(f_long) as sum1, avg(f_long)
as avg1, max(f_long) as max1, min(f_long) as min1, "
+ "sum(f_short) as sum2, avg(f_short)
as avg2, max(f_short) as max2, min(f_short) as min2, "
+ "sum(f_byte) as sum3, avg(f_byte)
as avg3, max(f_byte) as max3, min(f_byte) as min3, "
+ "sum(f_float) as sum4, avg(f_float)
as avg4, max(f_float) as max4, min(f_float) as min4, "
+ "sum(f_double) as sum5, avg(f_double)
as avg5, "
+ "max(f_double) as max5, min(f_double) as min5, "
+ "max(f_timestamp)
as max6, min(f_timestamp) as min6, "
+ "var_pop(f_double) as varpop1, var_samp(f_double) as
varsamp1, "
+ "var_pop(f_int) as varpop2, var_samp(f_int) as varsamp2 "
+ "FROM TABLE_A
group by f_int2";

```

or
```

String sql = "SELECT "
+ " order_id, site_id, count(*) as cnt "
+ "FROM ORDER_DETAILS
GROUP BY order_id, site_id"
+ ", TUMBLE(order_time, INTERVAL '1' HOUR) "
+ " UNION
SELECT "
+ " order_id, site_id, count(*) as cnt "
+ "FROM ORDER_DETAILS GROUP BY order_id,
site_id"
+ ", TUMBLE(order_time, INTERVAL '2' HOUR) ";

```

or
```

cli.execute(
"create table person (\n"
+ "id int COMMENT 'id', \n"
+
"name varchar(31) COMMENT 'name', \n"
+ "age int COMMENT 'age') \n"
+ "TYPE 'text'
\n"
+ "COMMENT '' LOCATION 'text://home/admin/orders'"
);

```

Imported from Jira [BEAM-3548](https://issues.apache.org/jira/browse/BEAM-3548). Original Jira may contain additional context.
Reported by: kedin.

Contributor guide

Open the contributing guide

Research direction

The issue names QueryBuilder, QueryTransform, and cli.execute, but no repository file or test. Start by locating the existing SQL APIs and their tests, then determine the scope needed for the fluent examples shown. Done means a documented fluid query builder can express the provided filtering, grouping, aggregation, union, and table-creation cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.