Project-MONAI / Project-MONAI/MONAILabel

OHIF deepgrow model gives Nonetype foreground interactions

Open
#1,654 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
891
Forks
269
Avg merge
15h 41m
Merged PRs (30d)
1

Description

Describe the bug
When using monailabel with OHIF I receive Nonetypes in my interactions on the z-axis. This happens specifically after correctly segmenting a sequence and moving to the next sequence in the same patient. Note, clearing points/interactions, making sure to remove previous segmentation, etc. did not seem to fix the issue. However, was able to fix the issue by refreshing the patient. Bug occurs independent of sample, i.e. if I refresh the patient I am able to segment the specific sequence.

Server logs
'foreground': [[114.18579234972675, 212.28415300546447, 10], [154.40437158469945, 174.1639344262295, 10], [171.54098360655735, 221.37704918032787, 10], [150.55737704918033, 225.224043715847, 13], [120.48087431693988, 162.62295081967213, 13], [178.88524590163934, 159.82513661202185, 13], [109.57377049180327, 179.0601092896175, None], [154.68852459016392, 178.36065573770492, None], [137.55191256830602, 202.49180327868854, None], [120.76502732240435, 176.2622950819672, None], [154.68852459016392, 187.80327868852459, None], [129.1584699453552, 207.3879781420765, None]], 'background': [], 'result_extension': '.nrrd', 'result_dtype': 'uint16', 'result_compress': False, 'description': 'Volumetric Interactivenet'}
[2024-03-15 13:24:41,491] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:76) - PRE - Run Transform(s)
[2024-03-15 13:24:41,491] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:77) - PRE - Input Keys: ['device', 'model', 'image', 'foreground', 'background', 'result_extension', 'result_dtype', 'result_compress', 'description', 'image_path']
[2024-03-15 13:24:41,520] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (LoadImaged): Time: 0.0278; image: torch.Size([320, 320, 15])(torch.float32)
[2024-03-15 13:24:41,521] [1] [MainThread] [ERROR] (uvicorn.error:434) - Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 276, in call
await super().call(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 92, in call
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 147, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in call
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 237, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
return await dependant.call(**values)
File "/usr/local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 180, in api_run_inference
return run_inference(background_tasks, model, image, session_id, params, file, label, output)
File "/usr/local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 162, in run_inference
result = instance.infer(request)
File "/usr/local/lib/python3.9/site-packages/monailabel/interfaces/app.py", line 307, in infer
result_file_name, result_json = task(request)
File "/usr/local/lib/python3.9/site-packages/monailabel/tasks/infer/basic_infer.py", line 295, in call
data = self.run_pre_transforms(data, pre_transforms)
File "/usr/local/lib/python3.9/site-packages/monailabel/tasks/infer/basic_infer.py", line 386, in run_pre_transforms
return run_transforms(data, transforms, log_prefix="PRE", use_compose=False)
File "/usr/local/lib/python3.9/site-packages/monailabel/interfaces/utils/transform.py", line 106, in run_transforms
data = t(data)
File "/usr/local/lib/python3.9/site-packages/monai/apps/deepgrow/transforms.py", line 571, in call
clicks = list(np.array(clicks, dtype=int))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

To Reproduce

  1. Foreground interactions on sequence X of patient Z
  2. Run deepgrow model
  3. Submit label
  4. Continue to sequence Y of patient Z (Note, in same patient)
    5a. (Clear points / previous segmentation map)
    5b. Create new segmentation map
  5. Foreground interactions on sequence Y of patient Z
  6. Run deepgrow model
  7. BUG -> Nonetype in foreground interactions

Expected behavior
Should be able to work on multiple sequences in the same patient.

Environment
pytorch=1.12.1+cu113
monai==1.2.0
monailabel==0.7.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at monailabel/endpoints/infer.py and monailabel/tasks/infer/basic_infer.py, then inspect the deepgrow transform shown in monai/apps/deepgrow/transforms.py. Reproduce the transition between two sequences for one patient using the listed steps and verify how foreground interactions are passed after clearing points and segmentation. Done means inference accepts valid interactions for multiple sequences without None values or the reported TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.