bloomberg / bloomberg/comdb2

Finish Semicolon Delimited Statement Support for cdb2sql

Open
#5,642 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
1.5k
Forks
243
Avg merge
2d 2h
Merged PRs (30d)
37

Description

# Project Context

Other database client CLIs allow statements to be delimited by semicolons. However, `cdb2sql` only allows statements to be delimited by newlines—with the exception of DDL statements, which use `$$` as a delimiter.

This is frustrating for developers who want to break long queries onto multiple lines for readability. Additionally, using newlines for most statements but `$$` for DDL is confusing and inconsistent.

**Goal:** Enable `cdb2sql` to support a conventional delimiter (like `;`) for all statements.
# Technical Description

A version of this feature was merged into our open source `cdb2sql` in [this PR](https://github.com/bloomberg/comdb2/pull/4938). The algorithm is shown below, with ⚠️ marking the lines that have limitations (explained in the next section).

```
for each line from client:
    if line is not a comment or empty:
        cdb2sql appends line to sql buffer
        while sql buffer contains a delimiter:
            cdb2sql sends sql to server
            if server finds a complete semicolon-delimited statement:  ⚠️
                server runs statement
                cdb2sql gets return code (if error, return to client and stop)
                cdb2sql receives end offset of executed statement from server  ⚠️
                cdb2sql trims executed statement from sql buffer
            else:
                cdb2sql receives 'incomplete' error from server
                break  # request next line from client

if sql buffer is non-empty after all input processed:
    cdb2sql returns error: incomplete statement
```

# Limitations

This feature does not work for SQL that bypasses the sqlite parser. Comdb2 uses sqlite's parser on most queries, but these statement types are handled separately (See [`handle_non_sqlite_requests`](https://github.com/bloomberg/comdb2/blob/4652c4d1579e305c96b342ae3ae542a7f65d7403/db/sqlinterfaces.c#L3683)):
- Stored procedure invocations
- Explain mode queries
- Expert mode queries

The two ⚠️ lines in the algorithm—detecting delimited statement boundaries and saving the offset of the end of the previously executed statement—are implemented *only* within the sqlite parser. If a statement doesn't go through sqlite, these operations fail and the algorithm has undefined behavior.

# Future Work

Deliver a fully functional version of this project by either implementing the missing algorithm components for the statement types that bypass the sqlite parser or limiting this feature to statement types that use the sqlite parser (returning a clear error for unsupported types).

Contributor guide

Open the contributing guide

Research direction

Start by reviewing PR 4938 and the statement handling around handle_non_sqlite_requests in db/sqlinterfaces.c. Trace how semicolon boundaries and executed-statement offsets are handled for SQLite-parsed statements versus stored procedures, explain mode, and expert mode. Done means the unsupported cases are either fully handled or rejected with a clear error.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sql, sqlite
Domain
cli, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.