HumanSignal / HumanSignal/label-studio-ml-backend
SAM 2 video backend does not actually do any work
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 490
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 3
Description
I recently installed SAM 2 video backend using docker but it does not do any work. When I click on label after creating bounding box on video, it shows loading icon right next to "Auto Label" toggle. and I see this log in docker terminal:
```
segment_anything_2_video | [2025-02-04 09:37:07,279] [DEBUG] [label_studio_ml.api::log_request_info::185] Request headers: Host: localhost:9090
segment_anything_2_video | Sentry-Trace: a3cb45b81e184a80931943254afad5ee-8ec0945f41c8c965-0
segment_anything_2_video | Baggage: sentry-environment=production,sentry-release=1.15.0,sentry-public_key=5f51920ff82a4675a495870244869c6b,sentry-trace_id=a3cb45b81e184a80931943254afad5ee,sentry-sample_rate=0.25,sentry-transaction=/projects/%3Aid%28%5Cd%2B%29/data,sentry-sampled=false
segment_anything_2_video | User-Agent: heartex/1.15.0+0.gdf34d55.dirty
segment_anything_2_video | Accept-Encoding: gzip, deflate
segment_anything_2_video | Accept: */*
segment_anything_2_video | Connection: keep-alive
segment_anything_2_video | Content-Length: 2065
segment_anything_2_video | Content-Type: application/json
segment_anything_2_video |
segment_anything_2_video |
segment_anything_2_video | [2025-02-04 09:37:07,280] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b'{"tasks": [{"id": 1, "annotations": [], "file_upload": "fd2d6f9a-test.mp4", "drafts": [{"id": 1, "user": "tmvkrpxl0@proton.me", "created_username": "tmvkrpxl0@proton.me, 1", "created_ago": "10\\u00a0minutes", "result": [{"value": {"framesCount": 228, "duration": 9.117318, "sequence": [{"frame": 25, "enabled": true, "rotation": 0, "x": 77.46875000000001, "y": 41.166666666666686, "width": 21.281250000000153, "height": 48.00000000000004, "time": 1}]}, "id": "Ms4o_fxlvh", "from_name": "box", "to_name": "video", "type": "videorectangle", "origin": "manual"}], "lead_time": 521.511, "was_postponed": false, "import_id": null, "created_at": "2025-02-04T09:26:19.805267Z", "updated_at": "2025-02-04T09:37:04.575509Z", "task": 1, "annotation": null}], "predictions": [], "data": {"video": "/data/upload/2/fd2d6f9a-test.mp4"}, "meta": {}, "created_at": "2025-02-04T09:25:33.762634Z", "updated_at": "2025-02-04T09:25:33.762649Z", "inner_id": 1, "total_annotations": 0, "cancelled_annotations": 0, "total_predictions": 0, "comment_count": 0, "unresolved_comment_count": 0, "last_comment_updated_at": null, "project": 2, "updated_by": null, "comment_authors": []}], "project": "2.1738657128", "label_config": "\\n \\n \\n \\n \\n \\n \\n \\n \\n \\n", "params": {"login": null, "password": null, "context": {"result": [{"value": {"framesCount": 228, "duration": 9.117318, "sequence": [{"frame": 25, "enabled": true, "rotation": 0, "x": 77.46875000000001, "y": 41.166666666666686, "width": 21.281250000000153, "height": 48.00000000000004, "time": 1}], "labels": ["Lego"]}, "id": "Ms4o_fxlvh", "from_name": "box", "to_name": "video", "type": "videorectangle", "origin": "manual"}]}}}'
segment_anything_2_video | [2025-02-04 09:37:07,280] [DEBUG] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::128] Image and upload dirs: image_dir=/root/.local/share/label-studio/media/upload, upload_dir=/root/.local/share/label-studio/media/upload
segment_anything_2_video | [2025-02-04 09:37:07,280] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::155] Resolving url using hostname [http://192.168.45.194:8080]: http://192.168.45.194:8080/data/upload/2/fd2d6f9a-test.mp4
segment_anything_2_video | [2025-02-04 09:37:07,281] [DEBUG] [model::predict::228] Video path: /root/.cache/label-studio/c49027ff__fd2d6f9a-test.mp4
segment_anything_2_video | [2025-02-04 09:37:07,281] [DEBUG] [model::get_prompts::102] Extracting keypoints from context: {'result': [{'value': {'framesCount': 228, 'duration': 9.117318, 'sequence': [{'frame': 25, 'enabled': True, 'rotation': 0, 'x': 77.46875000000001, 'y': 41.166666666666686, 'width': 21.281250000000153, 'height': 48.00000000000004, 'time': 1}], 'labels': ['Lego']}, 'id': 'Ms4o_fxlvh', 'from_name': 'box', 'to_name': 'video', 'type': 'videorectangle', 'origin': 'manual'}]}
segment_anything_2_video | [2025-02-04 09:37:07,281] [DEBUG] [model::predict::241] Number of prompts: 1, first frame index: 24, last frame index: 24, obj_ids: {'Ms4o_fxlvh': 0}
segment_anything_2_video | [2025-02-04 09:37:07,281] [DEBUG] [model::split_frames::61] Opening video file: /root/.cache/label-studio/c49027ff__fd2d6f9a-test.mp4
segment_anything_2_video | [2025-02-04 09:37:07,290] [DEBUG] [model::split_frames::69] Number of frames: 272
```
However it doesn't do anything after that. The docker container isn't frozen, it responds to blank enter press and whatnot. I can also send another request which also ends up not doing anything.
After a while the loading icon disappears on the webapp and no operation is applied to initial bounding box.
Is it because I used SAM 2.1? This is my docker-compose.yml file. I only changed model config and model name:
```yaml
version: "3.8"
services:
segment_anything_2_video:
container_name: segment_anything_2_video
image: humansignal/segment_anything_2_video:v0
build:
context: .
args:
TEST_ENV: ${TEST_ENV}
environment:
# specify these parameters if you want to use basic auth for the model server
- BASIC_AUTH_USER=
- BASIC_AUTH_PASS=
# set the log level for the model server
- LOG_LEVEL=DEBUG
# any other parameters that you want to pass to the model server
- ANY=PARAMETER
# specify the number of workers and threads for the model server
- WORKERS=1
- THREADS=8
# specify the model directory (likely you don't need to change this)
- MODEL_DIR=/data/models
# specify device
- DEVICE=cuda # or 'cpu' (coming soon)
# SAM2 model config
- MODEL_CONFIG=configs/sam2.1/sam2.1_hiera_l.yaml
# SAM2 checkpoint
- MODEL_CHECKPOINT=sam2.1_hiera_large.pt
# Specify the Label Studio URL and API key to access
# uploaded, local storage and cloud storage files.
# Do not use 'localhost' as it does not work within Docker containers.
# Use prefix 'http://' or 'https://' for the URL always.
# Determine the actual IP using 'ifconfig' (Linux/Mac) or 'ipconfig' (Windows).
- LABEL_STUDIO_URL=//something secret
- LABEL_STUDIO_API_KEY=//something secret2
ports:
- "9090:9090"
volumes:
- "./data/server:/data"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.