c/driver/postgresql: Handle non-COPY path results where individual string data would be larger than 2GB
- Dominant language
- C#
- Stars
- 627
- Forks
- 217
- Avg merge
- 17h
- Merged PRs (30d)
- 57
Description
### What feature or improvement would you like to see?
In the COPY reader we go to quite some lengths to ensure that we don't overflow the 2GB limit for the Arrow string type:
https://github.com/apache/arrow-adbc/blob/45cd9be77a619fe9b42f6b6470010ad430e5a868/c/driver/postgresql/statement.cc#L770-L776
This was not implemented in the PR that enabled converting PQresult objects to arrays (https://github.com/apache/arrow-adbc/pull/2029). While it is less likely that one of these queries will overflow (given that it is opt-in), we have the infrastructure to handle it and should!
Contributor guide
Research direction
Start in c/driver/postgresql/statement.cc at lines 770-776, then trace the PQresult-to-array conversion added by PR #2029 and compare it with the COPY reader's handling. Done means non-COPY results safely handle an individual string larger than 2GB without overflowing the Arrow string limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, postgresql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100