opensearch-project / opensearch-project/sql-odbc

[BUG] ODBC driver not properly paging

Open
#16 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
10
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
When using the ODBC driver from some SQL clients, only the first 200 rows are retrieved by default. Normally we would use a "LIMIT" clause to get more than 200 rows, but LIMIT is not supported by all SQL clients. We can manually increase the cluster opendistro.query.size_limit and the index.max_result_window settings, but this has significant memory overhead.

Can the ODBC driver support an efficient paging mechanism for these?

To Reproduce
Steps to reproduce the behavior:

  1. In SQL Server (or a different SQL client tool), add a linked server to the OpenSearch ODBC DSN:
    EXEC sp_addlinkedserver
    @server = N'OpenSearch',
    @srvproduct = N'',
    @provider = N'MSDASQL',
    @datasrc = N'OpenSearch SQL ODBC DSN';

  2. try to query the Linked Server using SQL (ensure the OpenSearch index has more than 200 rows)
    select top 1000 * from [OpenSearch]...[myopensearchindex]

What happens
only the first 200 rows are retrieved. We can force more rows by changing the cluster's opendistro.query.size_limit and each index's index.max_result_window --- but that is not optimal for memory.

Expected behavior
The ODBC driver should implement proper paging to get the data down to the client. Maybe it could honor the T-SQL "top" syntax instead of a "LIMIT" clause?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ODBC driver's query handling and reproduce the SQL Server linked-server query against an index with more than 200 rows. Done means queries requesting more than 200 rows retrieve them through efficient paging without requiring increased opendistro.query.size_limit or index.max_result_window settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.