Regeneration of DISABLE operations when setting instance operations
- Dominant language
- Java
- Stars
- 504
- Forks
- 255
- Avg merge
- 17d 47m
- Merged PRs (30d)
- 2
Description
### Describe the bug
There is a bug in the instance operation handling logic that causes an infinite cycle of regenerated DISABLE operations when applying operations like EVACUATE to an instance with HELIX_ENABLED=false in legacy fields. This prevents proper instance lifecycle transitions and causes unexpected behavior in instance swapping scenarios.
### To Reproduce
1. Create an instance in the cluster.
2. Set DISABLE operation with USER source:
Apply to InstanceConf...
`curl -X POST "http://:/admin/v2/clusters//instances/?command=setInstanceOperation&instanceOperationSource=USER&instanceOperation=DISABLE"`
3. Set EVACUATE operation with AUTOMATION source:
Apply to InstanceConf...
`curl -X POST "http://:/admin/v2/clusters//instances/?command=setInstanceOperation&instanceOperationSource=AUTOMATION&instanceOperation=EVACUATE"`
4. Set EVACUATE operation again with AUTOMATION source
5. Check the instance configuration in ZooKeeper
After the second EVACUATE operation, a new DISABLE operation is generated with:
1. Updated timestamp (more recent than the EVACUATE timestamp)
2. Added LEGACY_DISABLED_TYPE field
3. The cycle repeats with each subsequent evacuate operation
### Expected behavior
The DISABLE operation should not be regenerated with new timestamps when setting a different operation. Instance operations should be updated consistently without creating an infinite cycle.
### Additional context
RCA:
The regeneration happens because:
1. getActiveInstanceOperation() returns the last operation (EVACUATE)
2. EVACUATE is in INSTANCE_DISABLED_OVERRIDABLE_OPERATIONS
3. This causes getInstanceOperation() to override it with a newly generated DISABLE operation
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with getActiveInstanceOperation() and getInstanceOperation(), then reproduce the two EVACUATE requests through the setInstanceOperation API and inspect the instance configuration in ZooKeeper. Trace why EVACUATE causes a new DISABLE operation when HELIX_ENABLED=false in legacy fields. Done means repeated EVACUATE operations no longer regenerate DISABLE timestamps or LEGACY_DISABLED_TYPE entries, and instance transitions proceed normally.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100