Imageomics / Imageomics/pybioclip
Address embedding update with updated models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
BioCLIP 2.5 will be the new family of models (BioCLIP 2.5-H, BioCLIP 2.5-L, etc., where the letter references the CLIP model size).
- All these models are/will be trained on the revised TreeOfLife-200M, which includes an additional 19M images and more cleaning, the taxonomy remains un-changed.
- **Text embeddings will all be under model name folders** [here](https://huggingface.co/datasets/imageomics/TreeOfLife-200M/tree/main/embeddings).
- This is a breaking change for the pybioclip embed feature (as explained below).
- pybioclip default model will remain BioCLIP 2 because 2.5 is quite a bit larger, but it can still be used with pybioclip (we may update to 2.5-L, once that is ready).
The question for pybioclip is how we address a breaking change to the embed feature, since embeddings are a hard-coded pull from `embeddings/ txt_emb_species.npy` and `embeddings/ txt_emb_species.json` at the repo root (see [code](https://github.com/Imageomics/pybioclip/blob/main/src/bioclip/predict.py#L343)).
Related to #160
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/bioclip/predict.py around line 343, where the embed feature reads embeddings/ txt_emb_species.npy and embeddings/ txt_emb_species.json. Compare those hard-coded paths with the model-name folders in the TreeOfLife-200M embeddings dataset and review the BioCLIP 2.5 model naming described here. Done means pybioclip supports the revised layout without breaking the default BioCLIP 2 behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100