dimitri / dimitri/pgloader

pgloader fails parsing standard Ubuntu /etc/mysql/my.cnf containing !includedir

Open
#1,770 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Common Lisp
Stars
6.5k
Forks
613
Avg merge
3h 46m
Merged PRs (30d)
1

Description

Recent pgloader v3 builds fail on Ubuntu when /etc/mysql/my.cnf contains the standard MySQL !includedir directives.

The error is:

KABOOM!
MISSING-SECTION-HEADER-ERROR: Missing section header; found ! instead. at line 32

An unhandled error condition has been signalled:
   Missing section header; found ! instead. at line 32

This happens before the actual migration starts.

Environment

  • Ubuntu 24.04
  • MySQL 8.4
  • PostgreSQL 16
  • pgloader built from source:
pgloader version "3.6.ea152ef"
compiled with SBCL 2.2.9.debian

/etc/mysql/my.cnf

The standard Ubuntu MySQL configuration contains:

# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

In my case:

32  #
33  !includedir /etc/mysql/conf.d/
34  !includedir /etc/mysql/mysql.conf.d/

The reported line number corresponds to this area of /etc/mysql/my.cnf, not to the pgloader command file.

Reproduction

With a normal pgloader command file:

/usr/src/pgloader/build/bin/pgloader /root/migration.load

pgloader immediately exits with:

MISSING-SECTION-HEADER-ERROR: Missing section header; found ! instead. at line 32

There is no ! character anywhere in migration.load:

grep -n '!' /root/migration.load

returns no results.

Changing HOME does not help:

HOME=/root/pgloader-home \
  /usr/src/pgloader/build/bin/pgloader /root/migration.load

produces the same error.

Expected behavior

pgloader should either:

  1. support MySQL's !include / !includedir syntax when parsing /etc/mysql/my.cnf, or
  2. ignore those directives, or
  3. avoid parsing system-wide MySQL configuration files when all connection parameters are already supplied explicitly in the pgloader connection URI.

Why this is important

!includedir is part of the standard /etc/mysql/my.cnf shipped on Ubuntu/Debian installations, so automatically parsing this file with a generic INI parser makes current pgloader builds fail on a very common MySQL setup.

Older pgloader builds / the Mattermost pgloader Docker image do not exhibit this particular parsing error.

This appears to be related to the recently added automatic .my.cnf / /etc/mysql/my.cnf parsing in the v3 code path.

A simple fix may be to skip lines starting with !, similarly to how MySQL-specific include directives are handled elsewhere, or to implement support for !include and !includedir.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with /usr/src/pgloader/build/bin/pgloader /root/migration.load and the Ubuntu /etc/mysql/my.cnf containing !includedir. Trace the automatic .my.cnf and /etc/mysql/my.cnf parsing that runs before migration, then verify that these directives are handled or skipped and that an explicitly configured migration starts successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, postgresql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.