ashish10alex / ashish10alex/vscode-dataform-tools
bug: When using JS function I get this error: Syntax Error: Expected end of input but instead for WITH at [20:1]
- Dominant language
- TypeScript
- Stars
- 94
- Forks
- 16
- Avg merge
- 4m
- Merged PRs (30d)
- 7
Description
**Describe the bug**
I have many JS helper function in Dataform includes folder. If one of those functions takes an object of option parameters then it shows an error saying:
> Syntax Error: Expected end of input but instead for WITH at [20:1]
One example CTE code that causes this:
```SQLX
WITH ${common.buildIncrementalSliceCte({
cteName: "serp_rank_rows",
sourceRef: ref({ name: "firestore_website_serp_avg_daily_rank_by_year", schema: "mart" }),
tableAlias: "sm",
selectList: `
sm.gcp_project_id,
sm.collection_path,
sm.firestore_doc_id,
sm.firestore_doc_json,
sm.last_changed_at,
TO_HEX(SHA256(TO_JSON_STRING(sm.firestore_doc_json))) AS firestore_doc_json_hash,
"mart.firestore_website_serp_avg_daily_rank_by_year" AS source_table
`,
isIncremental: incremental(),
timestampColumn: "sm.last_changed_at",
dateColumn: "sm.last_changed_date",
watermarkVariable: "watermark_lower_bound",
})}
I tried to isolate the exact cause but it seems there is no common cause I can figure out.
To make it show populate a CTE with a Dataform JS function using an object of parameters, and make sure the option parameter list is large enough.
```
**To Reproduce**
Create helper similar to above and see the error when using.
**Expected behavior**
No error
**Environment:**
- Operating system: MacOs Sequoia 15.7.1
- Dataform cli version: 3.0.39
- Dataform core version: 3.0.39
- Version of the extension: 0.18.6
Contributor guide
Research direction
Start by reproducing the error with a Dataform JavaScript helper that accepts a sufficiently large options object, using the SQLX example and the listed Dataform CLI/core versions. Trace how the extension handles the generated SQLX and identify where the BigQuery-style WITH parse error appears; done means the example produces no syntax error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, javascript, sql, typescript, vscode
- Domain
- data-engineering, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100