tarantool / tarantool/migrations

Add is_healthy() check into up(). Now migrations will fail if patch_clusterwide is running

Open
#54 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
Lua
Stars
20
Forks
6
PR merge metrics
No merged PRs in 30d

Description

If clusterwide config has not applied yet after cluster start migration will fail.
This is a common scenario in tests.

See

code: 32
message: AtomicCallError: cartridge.patch_clusterwide is already running
stack traceback:
	/app/.rocks/share/tarantool/cartridge/twophase.lua:583: in function 'config_patch_clusterwide'
	/app/.rocks/share/tarantool/migrator.lua:105: in function 'up'
	eval:1: in main chunk
	[C]: at 0x006163c0

Maybe we can introduce is_healthy() check or a timeout into up() and wait until config is applied ?

Now i need to use such workaraound:

            if (!container.isRunning()) {
                container.start();
            }

            boolean healthy = false;
            int attempts = 30;
            while(!healthy && attempts-- >0) {
                List<?> result = container.executeCommand("return require('cartridge').is_healthy()").get();
                log.info("Checking cluster healthy status: {}, {}", result.size(), result.toString());
                if(result.size()==1) {
                    healthy = (Boolean)result.get(0);
                }
                Thread.sleep(1000);
            }

            if(!healthy) {
                throw new RuntimeException("Failed to get cluster in healthy state");
            }
            container.executeCommand("require('migrator').up()").get();

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the migrator.up entry point referenced in migrator.lua and the cluster-wide configuration flow shown in twophase.lua. Check how cartridge.is_healthy() can be used before migration; the work is done when up() avoids running during patch_clusterwide and handles failure to reach a healthy state.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.