HumanSignal / HumanSignal/label-studio
[bug] Resize handles and region selection often click-thru
- Dominant language
- TypeScript
- Stars
- 28.3k
- Forks
- 3.7k
- Avg merge
- 14h
- Merged PRs (30d)
- 15
Description
**Describe the bug**
In the image editor, clicking a Transformer resize handle or clicking a region to select it frequently seems to click-through or miss the resize handle. It either draws a new selection box or click-drags the underlying annotation rectangle. It's easy to tell this will happen because the cursor will not change to the resize-cursor. I've had this happen with polygon vertex handles as well, but I'm going to use the humble RectangleLabel in this issue.
When a rectangle annotation is selected, some of the 8 resize anchors work and others don't, and *which* ones work changes every time the same box is re-selected. I think this tends to happen on particularly large images (eg 10k+ pixels on a side).
It makes working on large images very frustrating.
**To Reproduce**
Steps to reproduce the behavior:
1. Create a label-studio project using the object-detection template.
2. Create a task with a large image, eg 15,000px x 15,000px
3. Create an annotation with RectangleLabels
4. Try to resize the RectangleLabels and notice only some of them afford the bi-directional-arrows resize-cursor
5. Note how re-selecting a RectangleLabel may change which handles are affected
6. More RectangleLabels per annotation and high zoom levels may increase likelihood of bad resize handles.
**Expected behavior**
Every resize handle grabs on the first click, consistently, across repeated re-selects.
**Screenshots**
_Faulty_ Handle Behavior (below)
_
_Functional_ Handle Behavior (below)
_
Video of Behavior
https://github.com/user-attachments/assets/71629c43-fa5c-482b-8d79-51e25dbbc8f3
**Environment (please complete the following information):**
- OS: Ubuntu Linux (X11)
- Browser: Brave 1.92.138 (64-bit) (Chromium: 150.0.7871.101). Hardware acceleration enabled.
- Label Studio Version: 1.23.0
**Additional context**
I believe this issue is related to #4558, #4452 and PR #9700 . These issues frame the problem of inconsistent handle click-behavior as the handles being too small. I don't think that's what's happening, as my experience shows that you don't have to be exactly on-mark to get the resize-cursor IF the resize-cursor is to appear at all. I think it has to do with konva's color-based hit-detection mechanism.
Contributor guide
Research direction
Reproduce the click-through behavior with the object-detection template, a 15,000px image, RectangleLabels, multiple annotations, and high zoom. Start at the Transformer resize handles and RectangleLabel selection behavior, then investigate the mentioned Konva color-based hit-detection mechanism. Done means every resize handle consistently shows the resize cursor and grabs on the first click after repeated re-selection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- frontend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100