Support remote Postgres instances
- Dominant language
- Python
- Stars
- 1k
- Forks
- 131
- PR merge metrics
- No merged PRs in 30d
Description
Unless I missing something, Odo currently relies on custom SQL via SQLAlchemy to execute a COPY statement that reads/writes from files local to the server. This may be disabled on some instances (eg AWS RDS), or not possible in a distributed environment where code can't run on the same instance as the source/target database.
https://github.com/blaze/odo/blob/master/odo/backends/sql_csv.py#L154
https://github.com/blaze/odo/blob/master/odo/backends/sql.py#L696
The Postgres COPY statement also supports reading/writing from stdin/stdout, which allows bulk reads/writes without requiring access to the local file system. The psycodb2 driver implements this through the cursor.copy_from/copy_to methods.
I'm not sure how to access this without going direct to the driver - eg not using SQLAlchemy - but doing so would allow Odo to be used in a much wider number of use cases (such as ours, where we have an ETL process bulk loading multiple different sources into a single target Postgres instance on AWS RDS.)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.