ashish10alex / ashish10alex/vscode-dataform-tools

bug: When using JS function I get this error: Syntax Error: Expected end of input but instead for WITH at [20:1]

Open
#242 7 comments 0 reactions 0 assignees View on GitHub
could-not-reproduce status:awaiting-response
Dominant language
TypeScript
Stars
94
Forks
16
Avg merge
4m
Merged PRs (30d)
7

Description

**Describe the bug**
I have many JS helper function in Dataform includes folder. If one of those functions takes an object of option parameters then it shows an error saying:

> Syntax Error: Expected end of input but instead for WITH at [20:1]

One example CTE code that causes this:

```SQLX
WITH ${common.buildIncrementalSliceCte({
cteName: "serp_rank_rows",
sourceRef: ref({ name: "firestore_website_serp_avg_daily_rank_by_year", schema: "mart" }),
tableAlias: "sm",
selectList: `
sm.gcp_project_id,
sm.collection_path,
sm.firestore_doc_id,
sm.firestore_doc_json,
sm.last_changed_at,
TO_HEX(SHA256(TO_JSON_STRING(sm.firestore_doc_json))) AS firestore_doc_json_hash,
"mart.firestore_website_serp_avg_daily_rank_by_year" AS source_table
`,
isIncremental: incremental(),
timestampColumn: "sm.last_changed_at",
dateColumn: "sm.last_changed_date",
watermarkVariable: "watermark_lower_bound",
})}

I tried to isolate the exact cause but it seems there is no common cause I can figure out.

To make it show populate a CTE with a Dataform JS function using an object of parameters, and make sure the option parameter list is large enough.

```

**To Reproduce**
Create helper similar to above and see the error when using.

**Expected behavior**
No error

**Environment:**
- Operating system: MacOs Sequoia 15.7.1
- Dataform cli version: 3.0.39
- Dataform core version: 3.0.39
- Version of the extension: 0.18.6

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the error with a Dataform JavaScript helper that accepts a sufficiently large options object, using the SQLX example and the listed Dataform CLI/core versions. Trace how the extension handles the generated SQLX and identify where the BigQuery-style WITH parse error appears; done means the example produces no syntax error.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, javascript, sql, typescript, vscode
Domain
data-engineering, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.