databrickslabs / databrickslabs/lsql
Keep same semantics for `mode="overwrite"` for StatementExecutionBackend
Open
Nobody has claimed this yet.
good first issue
- Dominant language
- Python
- Stars
- 39
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
we'd need to properly support overwrites in the raw sql shape to keep the same'ish semantics as Spark:
INSERT INTO {full_name}_tmp ...CREATE OR REPLACE TABLE {full_name} AS SELECT * FROM {full_name}_tmpDROP TABLE {full_name}_tmp
otherwise the failure of overwrite will leave the table in a corrupt state.
Originally posted by @nfx in https://github.com/databrickslabs/lsql/pull/74#discussion_r1541100575
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate StatementExecutionBackend and its handling of mode="overwrite" first. Compare the raw SQL path with the requested temporary-table, replacement, and cleanup sequence; done means overwrite failures do not leave the target table corrupted and the existing Spark-like semantics are preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100