opensearch-project / opensearch-project/sql-odbc
[BUG] ODBC driver not properly paging
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 10
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
Describe the bug
When using the ODBC driver from some SQL clients, only the first 200 rows are retrieved by default. Normally we would use a "LIMIT" clause to get more than 200 rows, but LIMIT is not supported by all SQL clients. We can manually increase the cluster opendistro.query.size_limit and the index.max_result_window settings, but this has significant memory overhead.
Can the ODBC driver support an efficient paging mechanism for these?
To Reproduce
Steps to reproduce the behavior:
-
In SQL Server (or a different SQL client tool), add a linked server to the OpenSearch ODBC DSN:
EXEC sp_addlinkedserver
@server = N'OpenSearch',
@srvproduct = N'',
@provider = N'MSDASQL',
@datasrc = N'OpenSearch SQL ODBC DSN'; -
try to query the Linked Server using SQL (ensure the OpenSearch index has more than 200 rows)
select top 1000 * from [OpenSearch]...[myopensearchindex]
What happens
only the first 200 rows are retrieved. We can force more rows by changing the cluster's opendistro.query.size_limit and each index's index.max_result_window --- but that is not optimal for memory.
Expected behavior
The ODBC driver should implement proper paging to get the data down to the client. Maybe it could honor the T-SQL "top" syntax instead of a "LIMIT" clause?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ODBC driver's query handling and reproduce the SQL Server linked-server query against an index with more than 200 rows. Done means queries requesting more than 200 rows retrieve them through efficient paging without requiring increased opendistro.query.size_limit or index.max_result_window settings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100