--migrate: Testcontainers IT suite (base + edge cases)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 970
- Forks
- 486
- Avg merge
- 3d 33m
- Merged PRs (30d)
- 170
Description
Part of #37237. Depends on the Ledger & verification, Core migration engine, and CLI surface tickets.
Summary
Migration-specific integration tests (in dotCMS/dot-ca-event-manager), extending the existing Testcontainers harness (2 ClickHouse replicas + Keeper + Toxiproxy, per .claude/testing.md). May land as two PRs (base cases, then edge cases) even though it's tracked as one ticket.
Scope
Base cases
Seed via docker/init/*.sql, apply a candidate migration through the real runner, assert:
- pre-existing queries are unaffected
- new columns/defaults are correct
- the schema change propagated to the second replica (poll
system.columns, the same shape as the existingassertReplicatedCount()helper) - applying the same migration twice is a no-op
Edge cases — grounded in the real bug already caught once during design (mutations_sync=2 breaking retry-safety), not hypothetical:
- a metadata-only
ADD COLUMNcase - a mutation-triggering case with
mutation_idcapture + poll to completion - checksum-mismatch aborts the run
V0-mismatch aborts the run- two concurrent runners contend for the Keeper lock and only one proceeds
- a script interrupted mid-way is safe to re-run in full
- a dropped connection during a long mutation (Toxiproxy) doesn't cause a retry to enqueue a duplicate mutation
- a
RENAME TABLEon a table with an attached refreshable MV, including one with aDEPENDS ONchain to another refreshable MV — three of the ten realenvironment-rollout runbooks (docs/migrations/) independently flag this as genuinely less battle-tested than a normal MV rename, not hypothetical caution
Verification (end-to-end, once this + all core tickets land)
Do a dry run of one real migration from the environment rollout (e.g. the simplest one, docs/migrations/07_engagement_daily.md) through the actual --migrate tool against a local stack, to confirm the tool works against genuine migration content, not just synthetic test SQL.
Repo
dotCMS/dot-ca-event-manager
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .claude/testing.md and the existing Testcontainers harness in dotCMS/dot-ca-event-manager, then inspect docker/init/*.sql and the assertReplicatedCount() helper. Implement the base and edge-case integration coverage described, including the real migration dry run using docs/migrations/07_engagement_daily.md once dependencies land. Done means the listed failure, retry, replication, locking, and refreshable-MV cases are verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, docker, java, sql
- Domain
- cli, databases, devops, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100