sequelize / sequelize/sequelize

Use COPY in bulkCreate for Postgres

Open
#15,404 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
30.4k
Forks
4.3k
Avg merge
1d 6h
Merged PRs (30d)
68

Description

Issue Creation Checklist

  • I understand that my issue will be automatically closed if I don't fill in the requested information
  • I have read the contribution guidelines

Feature Description

Describe the feature you'd like to see implemented

I'm creating 700k+ records with bulkCreate() and running into the Postgres server being killed by the OOM killer on Linux.

Ideally large inserts would be done with COPY instead of INSERT.

Describe why you would like this feature to be added to Sequelize

It will improve the performance of bulkCreate()

Is this feature dialect-specific?
  • No. This feature is relevant to Sequelize as a whole.
  • Yes. This feature only applies to the following dialect(s): Postgres, I am not familiar with other dialects.
Would you be willing to resolve this issue by submitting a Pull Request?
  • Yes, I have the time and I know how to start.
  • Yes, I have the time but I will need guidance.
  • No, I don't have the time, but my company or I are supporting Sequelize through donations on OpenCollective.
  • No, I don't have the time, and I understand that I will need to wait until someone from the community or maintainers is interested in implementing my feature.

Indicate your interest in the addition of this feature by adding the 👍 reaction. Comments such as "+1" will be removed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the bulkCreate entry point and the PostgreSQL dialect handling, then determine how COPY could support large inserts. Clarify the behavior and compatibility requirements before implementation. Done means large PostgreSQL bulkCreate operations use COPY where appropriate and retain expected Sequelize behavior, with tests covering the new path.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, typescript
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.