github / github/gh-ost

Move table mode for gh-ost

Open
#1,681 2 comments 0 reactions 1 assignee Claimed by @ericyan View on GitHub
Dominant language
Go
Stars
13.6k
Forks
1.4k
Avg merge
2h 31m
Merged PRs (30d)
4

Description

This document specifies the work needed inside gh-ost to support a new **move table mode**, in which one or more tables are copied from a source MySQL cluster to a target MySQL cluster, with a cooperative cutover protocol coordinated via hooks.

## Scope

**In scope:**

- One or more tables in a single gh-ost run, sharing one source binlog stream.
- GTID-only operation.
- gh-ost issues the source `RENAME TABLE` itself; ACL changes and feature-flag flips are delegated to hook scripts.

**Explicitly deferred or out of scope:**

- Schema transformation (`--alter`) combined with move.
- File+position binlog drain (non-GTID).
- Inline checksumming or target-binlog monitoring (use `pt-table-checksum` externally).
- Revert / rollback of a completed move.
- Source-side grant cleanup.

## Design

Move table mode is enabled by a single new flag `--move-tables=t1,t2,...`. The mode reuses the existing inspector, binlog streamer, chunk iterator, throttler, hook executor, and checkpoint mechanism, while introducing a second database connection (the target), per-table runtime state, and a cooperative cutover protocol driven by gh-ost.

### Architecture overview

```
┌──────────────────────┐ ┌──────────────────────┐
│ Source MySQL │ │ Target MySQL │
│ │ │ │
│ ┌─────────┐ │ │ ┌─────────┐ │
│ │ table_A │◄── Inspector reads ──┐ │ │ table_A │ │
│ │ table_B │ │ │ │ table_B │ │
│ └─────────┘ │ │ │ └─────────┘ │
│ │ │ │ ▲ │
│ Binlog ────────────────► EventsStreamer │ │ │
│ │ │ │ Applier writes ────┘
│ │ gh-ost │ │
│ │ process │ ┌─────────┐ │
│ │ │ │ │ _ghk │ │
│ │ └───────│──│checkpoint│ │
│ │ │ └─────────┘ │
└──────────────────────┘ └──────────────────────┘
```

**Key principle:** limit changes to the hot code path (eg. binlog streamer). All move-table-specific logic lives in higher layers — the migrator (cutover orchestration and multi-table iteration) and the applier (where it connects and what tables it writes to).

### Connection model

| Connection | Standard mode | Move table mode |
|---|---|---|
| Inspector | reads from source | reads from source (unchanged) |
| Binlog streamer | reads from source | reads from source (unchanged) |
| Applier | writes to source-side master | **writes to target master** |
| Cutover | issues `RENAME` on source | issues `RENAME` on source (short-lived, reuses inspector connection config) |

### Cutover protocol

The **read cutover** is the application repoints its reads to the target while writes still flow to the source. The **write cutover** is the actual "cutover", in which gh-ost issues a single atomic `RENAME TABLE` on the source, captures `@@gtid_executed` as the drain marker, waits until the applier has reached that GTID, and fires `on-success` so the hook can unlock the target write user and flip the application's write feature flag.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.