Upgraded sameersbn/gitlab from 9.2.7 to 11.1.4, won't start. DB errors

Please forgive my ignorance on this post. I'm new-ish to docker, gitlab, and databases.

I was able to update sameersbn/redis to the latest last night. I updated gitlab to the latest and was missing DB permissions. I finally got the db permissions fixed, but now the container exits.

I see this a bunch,

018-08-21 20:09:08,323 INFO exited: unicorn (exit status 1; not expected)
2018-08-21 20:09:09,326 INFO spawned: 'unicorn' with pid 743
2018-08-21 20:09:10,328 INFO success: unicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 20:09:10,531 INFO exited: unicorn (exit status 1; not expected)
2018-08-21 20:09:11,534 INFO spawned: 'unicorn' with pid 746
2018-08-21 20:09:12,536 INFO success: unicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 20:09:12,821 INFO exited: unicorn (exit status 1; not expected)
2018-08-21 20:09:13,824 INFO spawned: 'unicorn' with pid 749
2018-08-21 20:09:14,827 INFO success: unicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 20:09:15,048 INFO exited: unicorn (exit status 1; not expected)
2018-08-21 20:09:16,051 INFO spawned: 'unicorn' with pid 752
2018-08-21 20:09:17,054 INFO success: unicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Then a bit later,
018-08-21 19:41:42,707 INFO spawned: 'cron' with pid 631
2018-08-21 19:41:42,709 INFO spawned: 'nginx' with pid 632
2018-08-21 19:41:42,711 INFO spawned: 'sshd' with pid 633
2018-08-21 19:41:43,868 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 19:41:43,868 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 19:41:43,868 INFO success: unicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 19:41:43,869 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 19:41:43,869 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 19:41:43,869 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-08-21 19:41:43,869 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Missing Rails.application.secrets.openid_connect_signing_key for production environment. The secret will be generated and stored in config/secrets.yml.
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

Followed by A TON of what looks to be database data after the following text.

#<ActiveRecord::StatementInvalid: Mysql2::Error: Incorrect string value: '\xC2\x85 sen...' for column 'diff' at row 37: INSERT INTO merge_request_diff_files (diff, new_path, old_path, a_mode, b_mode, new_file, renamed_file, deleted_file, too_large, binary, merge_request_diff_id, relative_order)

Finally...

/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in _query' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in block in query'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in handle_interrupt' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in query'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/peek-mysql2-1.1.0/lib/peek/views/mysql2.rb:14:in query_with_timing' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_adapter.rb:484:in block in log' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activesupport-4.2.10/lib/active_support/notifications/instrumenter.rb:20:in instrument'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_adapter.rb:478:in log' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in execute'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/mysql2_adapter.rb:217:in execute' /home/git/gitlab/config/initializers/connection_fix.rb:20:in execute'
/home/git/gitlab/lib/gitlab/database.rb:164:in bulk_insert' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:80:in bulk_insert'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:70:in block in flush_buffers!' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in each'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in each_slice' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in each'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in flush_buffers!' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:44:in block in perform'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/delegation.rb:46:in each' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/delegation.rb:46:in each'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:33:in perform' /home/git/gitlab/lib/gitlab/background_migration.rb:49:in perform'
/home/git/gitlab/lib/gitlab/background_migration.rb:29:in block (2 levels) in steal' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.1.3/lib/sidekiq/api.rb:551:in block in each'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.1.3/lib/sidekiq/api.rb:550:in each' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.1.3/lib/sidekiq/api.rb:550:in each'
/home/git/gitlab/lib/gitlab/background_migration.rb:22:in block in steal' /home/git/gitlab/lib/gitlab/background_migration.rb:21:in each'
/home/git/gitlab/lib/gitlab/background_migration.rb:21:in steal' /home/git/gitlab/db/migrate/20171121135738_clean_up_from_merge_request_diffs_and_commits.rb:15:in up'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:611:in exec_migration' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:592:in block (2 levels) in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:591:in block in migrate' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in with_connection'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:590:in migrate' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:768:in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:1023:in block in execute_migration_in_transaction' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:1071:in ddl_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:1022:in execute_migration_in_transaction' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:984:in block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:980:in each' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:980:in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:823:in up' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:801:in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/tasks/database_tasks.rb:139:in migrate' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/railties/databases.rake:44:in block (2 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'

Caused by:
Mysql2::Error: Incorrect string value: '\xC2\x85 sen...' for column 'diff' at row 37
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in _query' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:120:in block in query'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in handle_interrupt' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/mysql2-0.4.10/lib/mysql2/client.rb:119:in query'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/peek-mysql2-1.1.0/lib/peek/views/mysql2.rb:14:in query_with_timing' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in block in execute'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_adapter.rb:484:in block in log' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activesupport-4.2.10/lib/active_support/notifications/instrumenter.rb:20:in instrument'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_adapter.rb:478:in log' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:317:in execute'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/mysql2_adapter.rb:217:in execute' /home/git/gitlab/config/initializers/connection_fix.rb:20:in execute'
/home/git/gitlab/lib/gitlab/database.rb:164:in bulk_insert' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:80:in bulk_insert'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:70:in block in flush_buffers!' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in each'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in each_slice' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in each'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:69:in flush_buffers!' /home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:44:in block in perform'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/delegation.rb:46:in each' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/delegation.rb:46:in each'
/home/git/gitlab/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb:33:in perform' /home/git/gitlab/lib/gitlab/background_migration.rb:49:in perform'
/home/git/gitlab/lib/gitlab/background_migration.rb:29:in block (2 levels) in steal' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.1.3/lib/sidekiq/api.rb:551:in block in each'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.1.3/lib/sidekiq/api.rb:550:in each' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.1.3/lib/sidekiq/api.rb:550:in each'
/home/git/gitlab/lib/gitlab/background_migration.rb:22:in block in steal' /home/git/gitlab/lib/gitlab/background_migration.rb:21:in each'
/home/git/gitlab/lib/gitlab/background_migration.rb:21:in steal' /home/git/gitlab/db/migrate/20171121135738_clean_up_from_merge_request_diffs_and_commits.rb:15:in up'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:611:in exec_migration' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:592:in block (2 levels) in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:591:in block in migrate' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in with_connection'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:590:in migrate' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:768:in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:1023:in block in execute_migration_in_transaction' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:1071:in ddl_transaction'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:1022:in execute_migration_in_transaction' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:984:in block in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:980:in each' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:980:in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:823:in up' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/migration.rb:801:in migrate'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/tasks/database_tasks.rb:139:in migrate' /home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.10/lib/active_record/railties/databases.rake:44:in block (2 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.4.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
root@dockerhost:~#

I can get in the gitlab container for about a minute before it exits and boots me out. I haven't seen anything in the log files in there that alarms me, but I will keep looking.

Any suggestions?

Start by reproducing the upgrade and tracing db/migrate/20171121135738_clean_up_from_merge_request_diffs_and_commits.rb through lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb. Inspect the MySQL error, config/initializers/connection_fix.rb, and the bulk insert path; done means the migration completes and the Docker container starts successfully.