Imageomics / Imageomics/pybioclip

Address embedding update with updated models

Open
#161 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
67
Forks
14
PR merge metrics
No merged PRs in 30d

Description

BioCLIP 2.5 will be the new family of models (BioCLIP 2.5-H, BioCLIP 2.5-L, etc., where the letter references the CLIP model size).
- All these models are/will be trained on the revised TreeOfLife-200M, which includes an additional 19M images and more cleaning, the taxonomy remains un-changed.
- **Text embeddings will all be under model name folders** [here](https://huggingface.co/datasets/imageomics/TreeOfLife-200M/tree/main/embeddings).
- This is a breaking change for the pybioclip embed feature (as explained below).
- pybioclip default model will remain BioCLIP 2 because 2.5 is quite a bit larger, but it can still be used with pybioclip (we may update to 2.5-L, once that is ready).

The question for pybioclip is how we address a breaking change to the embed feature, since embeddings are a hard-coded pull from `embeddings/ txt_emb_species.npy` and `embeddings/ txt_emb_species.json` at the repo root (see [code](https://github.com/Imageomics/pybioclip/blob/main/src/bioclip/predict.py#L343)).

Related to #160

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/bioclip/predict.py around line 343, where the embed feature reads embeddings/ txt_emb_species.npy and embeddings/ txt_emb_species.json. Compare those hard-coded paths with the model-name folders in the TreeOfLife-200M embeddings dataset and review the BioCLIP 2.5 model naming described here. Done means pybioclip supports the revised layout without breaking the default BioCLIP 2 behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.