HumanSignal / HumanSignal/label-studio

[bug] Resize handles and region selection often click-thru

Open
#9,810 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
28.3k
Forks
3.7k
Avg merge
14h
Merged PRs (30d)
15

Description

**Describe the bug**
In the image editor, clicking a Transformer resize handle or clicking a region to select it frequently seems to click-through or miss the resize handle. It either draws a new selection box or click-drags the underlying annotation rectangle. It's easy to tell this will happen because the cursor will not change to the resize-cursor. I've had this happen with polygon vertex handles as well, but I'm going to use the humble RectangleLabel in this issue.

When a rectangle annotation is selected, some of the 8 resize anchors work and others don't, and *which* ones work changes every time the same box is re-selected. I think this tends to happen on particularly large images (eg 10k+ pixels on a side).

It makes working on large images very frustrating.

**To Reproduce**
Steps to reproduce the behavior:
1. Create a label-studio project using the object-detection template.
2. Create a task with a large image, eg 15,000px x 15,000px
3. Create an annotation with RectangleLabels
4. Try to resize the RectangleLabels and notice only some of them afford the bi-directional-arrows resize-cursor
5. Note how re-selecting a RectangleLabel may change which handles are affected
6. More RectangleLabels per annotation and high zoom levels may increase likelihood of bad resize handles.

**Expected behavior**
Every resize handle grabs on the first click, consistently, across repeated re-selects.

**Screenshots**

_Faulty_ Handle Behavior (below)

Image

_

_Functional_ Handle Behavior (below)

Image

_

Video of Behavior

https://github.com/user-attachments/assets/71629c43-fa5c-482b-8d79-51e25dbbc8f3

**Environment (please complete the following information):**
- OS: Ubuntu Linux (X11)
- Browser: Brave 1.92.138 (64-bit) (Chromium: 150.0.7871.101). Hardware acceleration enabled.
- Label Studio Version: 1.23.0

**Additional context**
I believe this issue is related to #4558, #4452 and PR #9700 . These issues frame the problem of inconsistent handle click-behavior as the handles being too small. I don't think that's what's happening, as my experience shows that you don't have to be exactly on-mark to get the resize-cursor IF the resize-cursor is to appear at all. I think it has to do with konva's color-based hit-detection mechanism.

Contributor guide

Open the contributing guide

Research direction

Reproduce the click-through behavior with the object-detection template, a 15,000px image, RectangleLabels, multiple annotations, and high zoom. Start at the Transformer resize handles and RectangleLabel selection behavior, then investigate the mentioned Konva color-based hit-detection mechanism. Done means every resize handle consistently shows the resize cursor and grabs on the first click after repeated re-selection.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.