embulk / embulk/embulk-output-jdbc

ERROR: duplicate key violates unique constraint "pg_class_relname_nsp_index"

Open
#243 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
92
Forks
89
PR merge metrics
No merged PRs in 30d

Description

When running Embulk on multiple machines that writes to a single RedShift cluster, I often get the following error **(usually only on the first few runs)**:

`Error: java.lang.RuntimeException: org.postgresql.util.PSQLException: ERROR: duplicate key violates unique constraint "pg_class_relname_nsp_index" (possibly caused by concurrent transaction conflict)`

I found these two issues that seems related but both seems to be fixed in versions older than the one I'm using:
https://github.com/embulk/embulk-output-jdbc/issues/205 https://github.com/embulk/embulk-output-jdbc/issues/116

Embulk and plugin versions:
> Embulk v0.9.11
> embulk-input-mysql (0.9.3)
> embulk-output-redshift (0.8.1)

Contributor guide

No contributing guide indexed for this repository

Research direction

No source file or test is named. Start by reviewing related issues #205 and #116 and reproducing the failure with Embulk 0.9.11, embulk-input-mysql 0.9.3, and embulk-output-redshift 0.8.1 against one Redshift cluster from multiple machines; done means the concurrent first-run failure has a verified resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.