docker-library / docker-library/mysql

mysql 8.0.33 fails on Fly.io but works with 8.0.32

Open
#967 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
2.6k
Forks
2.2k
No merged PRs in 30d

Description

Hi!

I work at Fly.io, where we convert Docker images to Firecracker VM's. This has worked great for MySQL users until the current (as I write this) release of MySQL.

The current tag mysql:8 (which is mysql:8.0.33) fails during the Entrypoint execution, with no error message. However this works on my local Docker.

Here's what I see in version 8.0.33 output in Fly:

fly m run mysql:8.0.33 \
    -a some-fly-team --cpus 1 --memory 2048 --region ewr \
    -e "MYSQL_DATABASE=my_db" \
    -e "MYSQL_PASSWORD=secret" \
    -e "MYSQL_ROOT_PASSWORD=secret" \
    -e "MYSQL_USER=my_user"
👉 view log output of failing run (8.0.33) 
2023-04-18T00:30:06Z app[9185ee3c792383] ewr [info]Starting init (commit: ed64554)...
2023-04-18T00:30:06Z app[9185ee3c792383] ewr [info]Preparing to run: `docker-entrypoint.sh mysqld` as root
2023-04-18T00:30:06Z app[9185ee3c792383] ewr [info]2023/04/18 00:30:06 listening on [fdaa:1:40e4:a7b:21a1:b5fe:357b:2]:22 (DNS: [fdaa::3]:53)
2023-04-18T00:30:06Z app[9185ee3c792383] ewr [info]2023-04-18 00:30:06+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.33-1.el8 started.
2023-04-18T00:30:07Z app[9185ee3c792383] ewr [info]Starting clean up.
2023-04-18T00:30:08Z app[9185ee3c792383] ewr [info][    2.133634] reboot: Restarting system
2023-04-18T00:30:10Z app[9185ee3c792383] ewr [info]Starting init (commit: ed64554)...
2023-04-18T00:30:10Z app[9185ee3c792383] ewr [info]Preparing to run: `docker-entrypoint.sh mysqld` as root
2023-04-18T00:30:10Z app[9185ee3c792383] ewr [info]2023/04/18 00:30:10 listening on [fdaa:1:40e4:a7b:21a1:b5fe:357b:2]:22 (DNS: [fdaa::3]:53)
2023-04-18T00:30:10Z app[9185ee3c792383] ewr [info]2023-04-18 00:30:10+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.33-1.el8 started.
2023-04-18T00:30:11Z app[9185ee3c792383] ewr [info]Starting clean up.
2023-04-18T00:30:12Z app[9185ee3c792383] ewr [info][    2.132792] reboot: Restarting system
2023-04-18T00:30:13Z app[9185ee3c792383] ewr [info]Starting init (commit: ed64554)...
2023-04-18T00:30:13Z app[9185ee3c792383] ewr [info]Preparing to run: `docker-entrypoint.sh mysqld` as root
2023-04-18T00:30:13Z app[9185ee3c792383] ewr [info]2023/04/18 00:30:13 listening on [fdaa:1:40e4:a7b:21a1:b5fe:357b:2]:22 (DNS: [fdaa::3]:53)
2023-04-18T00:30:13Z app[9185ee3c792383] ewr [info]2023-04-18 00:30:13+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.33-1.el8 started.
2023-04-18T00:30:14Z app[9185ee3c792383] ewr [info]Starting clean up.
2023-04-18T00:30:15Z app[9185ee3c792383] ewr [info][    2.138144] reboot: Restarting system
2023-04-18T00:30:16Z app[9185ee3c792383] ewr [info]2023/04/18 00:30:16 listening on [fdaa:1:40e4:a7b:21a1:b5fe:357b:2]:22 (DNS: [fdaa::3]:53)
2023-04-18T00:30:16Z app[9185ee3c792383] ewr [info]2023-04-18 00:30:16+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.33-1.el8 started.
2023-04-18T00:30:17Z app[9185ee3c792383] ewr [info]Starting clean up.
2023-04-18T00:30:18Z app[9185ee3c792383] ewr [info][    2.135679] reboot: Restarting system
2023-04-18T00:30:21Z app[9185ee3c792383] ewr [info]Starting clean up

The entrypoint script is exiting with some non-zero status code, and Fly is rebooting it. It goes until it hits the max restart limit (defaulting to 10).

Version 8.0.32 does not do this, and works as you'd expect:

fly m run mysql:8.0.32 \
    -a some-fly-team --cpus 1 --memory 2048 --region ewr \
    -e "MYSQL_DATABASE=my_db" \
    -e "MYSQL_PASSWORD=secret" \
    -e "MYSQL_ROOT_PASSWORD=secret" \
    -e "MYSQL_USER=my_user"
👉 view log output of successful run (8.0.32) 
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]Starting init (commit: ed64554)...
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]Preparing to run: `docker-entrypoint.sh mysqld` as root
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023/04/18 00:29:02 listening on [fdaa:1:40e4:a7b:ce:414e:3b11:2]:22 (DNS: [fdaa::3]:53)
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18 00:29:02+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.32-1.el8 started.
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18 00:29:02+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18 00:29:02+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.32-1.el8 started.
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18 00:29:02+00:00 [Note] [Entrypoint]: Initializing database files
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18T00:29:02.659391Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18T00:29:02.659604Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.32) initializing of server in progress as process 591
2023-04-18T00:29:02Z app[91857775f1d308] ewr [info]2023-04-18T00:29:02.667692Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-04-18T00:29:03Z app[91857775f1d308] ewr [info]2023-04-18T00:29:03.049568Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2023-04-18T00:29:03Z app[91857775f1d308] ewr [info]2023-04-18T00:29:03.986375Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2023-04-18T00:29:05Z app[91857775f1d308] ewr [info]2023-04-18 00:29:05+00:00 [Note] [Entrypoint]: Database files initialized
2023-04-18T00:29:05Z app[91857775f1d308] ewr [info]2023-04-18 00:29:05+00:00 [Note] [Entrypoint]: Starting temporary server
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.095803Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.098734Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.32) starting as process 633
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.112757Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.363615Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.598917Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to suppo
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.598917Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.599453Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.612631Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2023-04-18T00:29:06Z app[91857775f1d308] ewr [info]2023-04-18T00:29:06.612803Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.32'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.

I'd love some help in seeing what changed or may be failing here! In theory whatever works in Docker should work in a Fly VM...

Is the only notable change the update of the Gosu version? https://github.com/docker-library/mysql/commit/99f9c4104edb05ce198c5e9282f4d9d702277a60

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docker-entrypoint.sh and compare the MySQL 8.0.32 and 8.0.33 image behavior using the provided Fly.io logs. Review the linked Gosu update and determine what causes the 8.0.33 entrypoint to exit and reboot under Fly while succeeding in local Docker; done means the regression is explained and a tested compatible fix or recommendation is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, mysql, shell
Domain
databases, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.