streaming table created via sdk gets removed after 30 minutes

Open
#524 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
sql, typescript
Domain
api, database

Research direction

Start by reproducing a CREATE OR REFRESH STREAMING TABLE submission through the Node.js SDK and compare its behavior with the same command in the Databricks SQL editor. Investigate why the SDK-created table disappears after 30 minutes despite no DROP statement; done means identifying the cause and verifying that an SDK-created streaming table persists.

Written by the indexing model from the issue text.

Description

i am building an integration where i am submitting SQL to databricks via this SDK.

when i submit a CREATE OR REFRESH STREAMING TABLE command, it completes... but 30 minutes later, it just disappears from databricks completely.

when i create streaming tables via the sql editor in the databricks console, it seems to stick around, no problem.

i triple checked, and there are no DROP statements, or anything that we are doing to remove the streaming table.

here is some output from genie about this issue -

Image
Dominant language
TypeScript
Stars
36
Forks
50
Avg merge
13h 46m
Merged PRs (30d)
9

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/databricks-sql-nodejs

All issues in databricks/databricks-sql-nodejs

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.