Project-MONAI / Project-MONAI/model-zoo
Input image transforms for some of the models may be improved
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 338
- Forks
- 106
- PR merge metrics
- No merged PRs in 30d
Description
Is your feature request related to a problem? Please describe.
The input image transforms in some of the models may not be configured to offer optimal input to the model during training, validation, and inference.
For instance the endoscopic_inbody_classification example:
- uses a
Resizedtransform that shrinks images to 256×256 pixels, but it does not enableanti_aliasing. Especially when downscaling large video frames with sharp details, patterns, or lines in them, this can lead to aliased artifacts that may cause the model to learn the wrong things or recognize structures that aren't really in the original image. - uses
NormalizeIntensitydwithnonzeroset totrue. If there are zero-valued pixels in the image, they will not be scaled and offset together with the rest, which may cause discontinuities and again make it seem as if there are structures that aren't really there. - uses
NormalizeIntensitydwithout specifying a fixed subtrahend and divisor. This means the intensities in each image will be normalized according to the mean and standard deviation in that specific image. If the image only contains a narrow range of intensities, for instance a dark image with sensor noise, this will be blown up to a big noisy mess in which the model might recognize random things. The usual way of working for ImageNet and such, is to calculate the mean and stddev across the entire training set and use those values everywhere.
Describe the solution you'd like
- Set
"anti_aliasing": truein theResizedtransforms. - Consider leaving
nonzeroatfalsein theNormalizeIntensitydtransform unless there really is a good reason for it. - Consider setting a fixed subtrahend and divisor in the
NormalizeIntensitydtransform. - Re-train models with whatever parameters were changed.
Additional context
A paper that discusses the impact of aliasing in convolutional networks
I have tested the impact on processing time of enabling anti_aliasing. On a GPU (RTX A2000), the impact is tiny: transforming a 720p video frame and adding it to a batch takes 11ms instead of 9. On CPU the impact is much larger (55ms instead of 9).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the endoscopic_inbody_classification example and inspect its Resized and NormalizeIntensityd transforms. Determine which other model examples use the same settings, decide the fixed normalization statistics where needed, and retrain models affected by the changes; completion means the transforms use the agreed parameters and updated models are available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100