OHDSI / OHDSI/DatabaseConnector
Running out of memory when pushing Andromeda tables to external databases using bulk import
Open
Nobody has claimed this yet.
bug
- Dominant language
- R
- Stars
- 57
- Forks
- 93
- Avg merge
- 10d 12h
- Merged PRs (30d)
- 1
Description
Noticed while trying to push the concept_relationship table to Postgres. It probably has to with writing the Andromeda table to CSV prior to calling the PG bulk upload tool
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the attempt to push the concept_relationship Andromeda table to Postgres using bulk import. Inspect the path that writes the Andromeda table to CSV before calling the PostgreSQL bulk upload tool. Done means the import completes without running out of memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, r
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100