Project-MONAI / Project-MONAI/MONAILabel
OHIF deepgrow model gives Nonetype foreground interactions
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 891
- Forks
- 269
- Avg merge
- 15h 41m
- Merged PRs (30d)
- 1
Description
Describe the bug
When using monailabel with OHIF I receive Nonetypes in my interactions on the z-axis. This happens specifically after correctly segmenting a sequence and moving to the next sequence in the same patient. Note, clearing points/interactions, making sure to remove previous segmentation, etc. did not seem to fix the issue. However, was able to fix the issue by refreshing the patient. Bug occurs independent of sample, i.e. if I refresh the patient I am able to segment the specific sequence.
Server logs
'foreground': [[114.18579234972675, 212.28415300546447, 10], [154.40437158469945, 174.1639344262295, 10], [171.54098360655735, 221.37704918032787, 10], [150.55737704918033, 225.224043715847, 13], [120.48087431693988, 162.62295081967213, 13], [178.88524590163934, 159.82513661202185, 13], [109.57377049180327, 179.0601092896175, None], [154.68852459016392, 178.36065573770492, None], [137.55191256830602, 202.49180327868854, None], [120.76502732240435, 176.2622950819672, None], [154.68852459016392, 187.80327868852459, None], [129.1584699453552, 207.3879781420765, None]], 'background': [], 'result_extension': '.nrrd', 'result_dtype': 'uint16', 'result_compress': False, 'description': 'Volumetric Interactivenet'}
[2024-03-15 13:24:41,491] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:76) - PRE - Run Transform(s)
[2024-03-15 13:24:41,491] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:77) - PRE - Input Keys: ['device', 'model', 'image', 'foreground', 'background', 'result_extension', 'result_dtype', 'result_compress', 'description', 'image_path']
[2024-03-15 13:24:41,520] [1] [MainThread] [INFO] (monailabel.interfaces.utils.transform:122) - PRE - Transform (LoadImaged): Time: 0.0278; image: torch.Size([320, 320, 15])(torch.float32)
[2024-03-15 13:24:41,521] [1] [MainThread] [ERROR] (uvicorn.error:434) - Exception in ASGI application
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
return await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 276, in call
await super().call(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in call
await self.middleware_stack(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in call
raise exc
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in call
await self.app(scope, receive, _send)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 92, in call
await self.simple_response(scope, receive, send, request_headers=headers)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 147, in simple_response
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in call
raise exc
File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in call
await self.app(scope, receive, sender)
File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call
raise e
File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in call
await route.handle(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 237, in app
raw_response = await run_endpoint_function(
File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
return await dependant.call(**values)
File "/usr/local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 180, in api_run_inference
return run_inference(background_tasks, model, image, session_id, params, file, label, output)
File "/usr/local/lib/python3.9/site-packages/monailabel/endpoints/infer.py", line 162, in run_inference
result = instance.infer(request)
File "/usr/local/lib/python3.9/site-packages/monailabel/interfaces/app.py", line 307, in infer
result_file_name, result_json = task(request)
File "/usr/local/lib/python3.9/site-packages/monailabel/tasks/infer/basic_infer.py", line 295, in call
data = self.run_pre_transforms(data, pre_transforms)
File "/usr/local/lib/python3.9/site-packages/monailabel/tasks/infer/basic_infer.py", line 386, in run_pre_transforms
return run_transforms(data, transforms, log_prefix="PRE", use_compose=False)
File "/usr/local/lib/python3.9/site-packages/monailabel/interfaces/utils/transform.py", line 106, in run_transforms
data = t(data)
File "/usr/local/lib/python3.9/site-packages/monai/apps/deepgrow/transforms.py", line 571, in call
clicks = list(np.array(clicks, dtype=int))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
To Reproduce
- Foreground interactions on sequence X of patient Z
- Run deepgrow model
- Submit label
- Continue to sequence Y of patient Z (Note, in same patient)
5a. (Clear points / previous segmentation map)
5b. Create new segmentation map - Foreground interactions on sequence Y of patient Z
- Run deepgrow model
- BUG -> Nonetype in foreground interactions
Expected behavior
Should be able to work on multiple sequences in the same patient.
Environment
pytorch=1.12.1+cu113
monai==1.2.0
monailabel==0.7.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at monailabel/endpoints/infer.py and monailabel/tasks/infer/basic_infer.py, then inspect the deepgrow transform shown in monai/apps/deepgrow/transforms.py. Reproduce the transition between two sequences for one patient using the listed steps and verify how foreground interactions are passed after clearing points and segmentation. Done means inference accepts valid interactions for multiple sequences without None values or the reported TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100