Imageomics / Imageomics/FuncaPalooza-2025
BioCLIP v2 orthogonal dims
- Dominant language
- No language data
- Stars
- 5
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
I personally would like to play around with the new and improved [BioCLIPv2](https://arxiv.org/pdf/2505.23883) embedding space. Specifically, if we can use the emergent properties of the embeddings for 0 shot or few shot classifiers or trait quantification.
From the CS side this is two ideas:
- Few-shot Classifier Ablations: Can we define actionable recommendations for creating "biologist ready" image classifiers for a "beyond species ID" task from BioCLIP 2 embeddings? Curious of data needs for different deployments, normalization, augmentation etc. Could apply on a novel task for understanding in a new biological context, and capture an interesting pattern across a different gradient. One idea could be for plants that have highly variable growth forms, and if we can quantify patterns in selection of growth form across large scales using inat or other image sources.
- Fine-grain explainability of orthogonal embedding dimensions for trait discovery or localization. [FinerCam](https://arxiv.org/pdf/2501.11309) can localize in-image features that cause class distinctions between two images. Is there a method we can develop that can localize image features that cause stratification along an emergent embedding axis other than species ID? For instance if a juvenile adult axis exists in the embedding space, can we show what image features cause separation of two samples along that access?
From the Bio side, I am open to whatever, but if it were up to me would like to work with phenology data. Can we create species agnostic classifiers of phenophase that could potentially be used to detect structures that inform that prediction?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the linked BioCLIPv2 and FinerCam papers, then narrow the proposal to either few-shot classifier ablations or localization of embedding axes. The issue does not identify repository files, tests, or an implementation entry point; done would require a defined experiment, biological task, evaluation plan, and documented results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- machine-learning
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100