lllyasviel / lllyasviel/ControlNet
Can ControlNet be used to do virtual staining of hisopathological (H&E) images?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
I was wondering if ControlNet could be used to virtually stain an unstained image for my research purposes. For those not aware of how unstained and stained H&E images look like, they look like [this[(https://www.google.com/search?sxsrf=AJOqlzWGwEsKcZEGMowYq7gSsUmbJNS3VQ:1675370125229&q=unstained+to+stained+h%26e+image&tbm=isch&sa=X&ved=2ahUKEwj1ibmJ2Pf8AhUTUzUKHRiHCwcQ0pQJegQIDxAB&biw=1873&bih=969&dpr=1#imgrc=x3HbfcorYe-2yM). The unstained and stained images would have the same structure & texture (essentially the same segmentation mask) but different color obviously. I was wondering if with enough training images, the model could accurately stain the unstained images to make them look like real H&E stained images.
Would appreciate any input, thanks a lot in advance!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by reviewing ControlNet's conditioning and training documentation, then assess whether paired unstained and H&E images could support the proposed virtual-staining experiment; done means a documented feasibility answer.
Written by the indexing model from the issue text.
Assessment
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100