dimitri / dimitri/pgloader

Regression on pgloader v4: uppercase MySQL column names cause COPY/INDEX failures in data-only mode

Open
#1,772 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Common Lisp
Stars
6.5k
Forks
613
Avg merge
3h 46m
Merged PRs (30d)
1

Description

Description

When migrating from MySQL to PostgreSQL using data-only mode, pgloader v4 fails if MySQL columns have uppercase names (e.g. MY_COLUMN) while the corresponding PostgreSQL columns are lowercase (my_column). This worked correctly in pgloader v3, where column name casing was automatically normalized during migration.

Steps to Reproduce

  1. Create a MySQL table with uppercase column names (e.g. MY_COLUMN, ANOTHER_COL)
  2. Create the corresponding PostgreSQL table with lowercase column names (my_column, another_col)
  3. Run pgloader v4 in data-only mode to migrate data from MySQL to PostgreSQL.

Expected Behavior
pgloader should normalize column name casing when mapping MySQL columns to PostgreSQL columns, as it did in v3.

Actual Behavior

 pgloader v4 uses the MySQL column names as-is (uppercase), resulting in errors:
 ERROR: column "MY_COLUMN" does not exist
  COPY init failed: ERROR: syntax error at or near ")"
  Failed to re-enable triggers: current transaction is aborted, commands ignored until end of transaction block

INDEX and PK INDEX creation also fails with the same column-not-found error. The entire table migration is aborted.

Why quoting is not a viable workaround?

On PostgreSQL, using double-quoted identifiers (e.g. "MY_COLUMN") to preserve uppercase naming is not a practical solution. Quoted identifiers make PostgreSQL treat column names as case-sensitive, which breaks existing application queries that reference columns without quotes.
Any query using SELECT my_column FROM ... would fail because PostgreSQL would no longer match it to "MY_COLUMN". This forces changes across the entire application layer, which is unacceptable for production migrations.

Environment

  • Source: MySQL 8.0
  • Target: PostgreSQL (lowercase column convention)
  • pgloader: v4 (Clojure rewrite)
  • Mode: data-only migration

Notes
This is a regression from pgloader v3, which handled case-mismatched columns without issue. Since v4 is a complete rewrite in Clojure, the column name normalization logic from v3 appears to be missing.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the data-only migration with uppercase MySQL columns and lowercase PostgreSQL columns, then compare the v4 behavior with pgloader v3. Trace the v4 column mapping and COPY, INDEX, and primary-key handling to find where normalization is missing. Done means the migration completes and data, indexes, and primary keys are created successfully without requiring quoted uppercase identifiers.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, postgresql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.