linkedin / linkedin/coral

Does Trino to Spark only support conversion syntax differences? Does it not support conversion of function differences?

Open
#413 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
906
Forks
219
Avg merge
2d 13h
Merged PRs (30d)
2

Description

I tried to convert these two functions, but both resulted in an error message "No match found for function".
`select regexp_like('a','\d')`
`select date_diff('month',cast('2022-05-23' as date),cast('2022-06-23' as date))`
![733A80B6-9BE2-4829-8747-5B6A6C557A9F](https://github.com/linkedin/coral/assets/31527268/126bf2be-c2c2-457f-9e6f-fcccea7ebf19)

When I try to convert SQL statements with syntax differences, Coral can successfully perform the conversion, for example:
`SELECT *
FROM mydb.mytable AS a
CROSS JOIN UNNEST(split(a.id, ',')) AS t0 (b)`

May I ask if the conversion of Trino to Spark function differences is not currently supported?
I see validator. validate (query) in the code
It is verified using the rules of hive, and the unique function in trino will report an error at this step

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at validator.validate(query) and the Hive validation rules mentioned in the report. Reproduce the regexp_like and date_diff queries, then compare their validation path with the working CROSS JOIN UNNEST conversion. Done should establish whether Trino-to-Spark function differences are supported and identify the relevant conversion or validation boundary.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark, sql
Domain
data-engineering, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.