confluentinc / confluentinc/ksql

CTAS: Erroneous "Duplicate values found in schema"

Open
#3,976 1 comment 0 reactions 0 assignees View on GitHub
bug engine
Dominant language
Java
Stars
315
Forks
1k
Avg merge
1d 7h
Merged PRs (30d)
36

Description

**Describe the bug**

The assignment of internal column names is not consistent across different queries. The problem might be in the auto-increment when creating a new internal column name.

**To Reproduce**
Steps to reproduce the behavior, include:
1. The version of KSQL.
Master
2. Sample source data.
No need for source data
3. Any SQL statements you ran

First create a table with a value column without any alias. Then, try to create a new table that uses this column again without an alias.

```
describe metrics_cube;

Name : METRICS_CUBE
Field | Type
---------------------------------------------------
ROWTIME | BIGINT (system)
METRICS_STREAM.ROWKEY | VARCHAR(STRING)
METRICS_NAME | VARCHAR(STRING)
METRICS_VALUE | DOUBLE
KSQL_COL_2 | ARRAY
---------------------------------------------------
For runtime statistics and query details run: DESCRIBE EXTENDED ;
```

```
CREATE TABLE metrics_table AS
SELECT metrics_name, KSQL_COL_2, avg(metrics_value) from metrics_cube WINDOW TUMBLING (size 60 seconds)
GROUP BY metrics_name, KSQL_COL_2;
```

Gives the error
`Duplicate values found in schema: `KSQL_COL_2` DOUBLE`

**Expected behavior**
Assign a new internal column name `KSQL_COL_3` to the non-aliased column

**Actual behaviour**
A clear and concise description of what actually happens, including:
1. CLI output
2. Error messages
`Duplicate values found in schema: `KSQL_COL_2` DOUBLE`
3. KSQL logs

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start by reproducing the supplied CTAS statements against the described schema and trace the internal column-name assignment for the non-aliased KSQL_COL_2 reference. Done means the second table is created without a duplicate-schema error and the column receives the expected KSQL_COL_3 name.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases, stream-processing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.