Project-MONAI / Project-MONAI/MONAI
Implementing foundation models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.7k
- Forks
- 1.6k
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 20
Description
Is your feature request related to a problem? Please describe.
I have been using MONAI for my project the last few months, the main problem is I need to use publicly available foundation models such as RAFT or MiDaS to generate intermidiate inputs for my models. This makes the implementation of such networks a bit tricky as you have to stick to certain input requirements (size, normalization, etc.) and implement those outside the MONAI bundles.
Describe the solution you'd like
Implementing a set of foundation models for MONAI. For my own application, I need depth estimation and optical flow estimation. However, I think it will benefitial to add models for segmentation and object detection also. I think there are two possible solutions:
- Create native foundation models in MONAI for these tasks.
- "Porting" some of the general foundation models available in Pytorch such as RAFT or MiDaS and implement them inside the MONAI framework.
Additional context
I'd like to use this post as a discussion about the implementation of such foundation models. I am more than happy to participate on the process, as it would massively benefit my own work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points and presents foundation-model support as an open discussion. First narrow the scope to specific models and tasks, then define the integration point and acceptance criteria before implementation can be considered complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100