sequelize / sequelize/sequelize
Use COPY in bulkCreate for Postgres
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 30.4k
- Forks
- 4.3k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 68
Description
Issue Creation Checklist
- I understand that my issue will be automatically closed if I don't fill in the requested information
- I have read the contribution guidelines
Feature Description
Describe the feature you'd like to see implemented
I'm creating 700k+ records with bulkCreate() and running into the Postgres server being killed by the OOM killer on Linux.
Ideally large inserts would be done with COPY instead of INSERT.
Describe why you would like this feature to be added to Sequelize
It will improve the performance of bulkCreate()
Is this feature dialect-specific?
- No. This feature is relevant to Sequelize as a whole.
- Yes. This feature only applies to the following dialect(s): Postgres, I am not familiar with other dialects.
Would you be willing to resolve this issue by submitting a Pull Request?
- Yes, I have the time and I know how to start.
- Yes, I have the time but I will need guidance.
- No, I don't have the time, but my company or I are supporting Sequelize through donations on OpenCollective.
- No, I don't have the time, and I understand that I will need to wait until someone from the community or maintainers is interested in implementing my feature.
Indicate your interest in the addition of this feature by adding the 👍 reaction. Comments such as "+1" will be removed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the bulkCreate entry point and the PostgreSQL dialect handling, then determine how COPY could support large inserts. Clarify the behavior and compatibility requirements before implementation. Done means large PostgreSQL bulkCreate operations use COPY where appropriate and retain expected Sequelize behavior, with tests covering the new path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, typescript
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100