[VL] Sql Function `map` does not throw exception when there are duplicate keys and spark.sql.mapKeyDedupPolicy=EXCEPTION.
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
This should throw exception if spark.sql.mapKeyDedupPolicy=EXCEPTION (spark's default)
```sql
create table fun_map_src(k1 int, v1 int, k2 int, v2 int);
insert into fun_map_src values (1, 3, 1, 4);
select map(k1, v1, k2, v2) from fun_map_src;
```
However, when gluten enabled, the output is
```sql
> select map(k1, v1, k2, v2) from fun_map_src;
25/02/12 17:48:00 WARN WholeStageTransformer: WholeStageCodegenTransformer (1) generating the substrait plan took: 7 ms.
I20250212 17:48:01.671165 386506 Task.cpp:2051] Terminating task Gluten_Stage_0_TID_0_VTID_0 with state Finished after running for 25ms
{1:4}
```
(This look like LAST_WIN?)
> Note, You should select from the table, not the constant value such as `select from Map(1, 3, 1, 4);`, because the in that way, the exception is from the plan phase.
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start by running the reported table-based SQL reproduction with spark.sql.mapKeyDedupPolicy=EXCEPTION, rather than the constant-map form noted in the issue. Trace the Velox execution path for map(k1, v1, k2, v2) and compare its duplicate-key handling with Spark’s expected behavior. Done means the query throws an exception instead of returning {1:4}.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100