Imageomics / Imageomics/FuncaPalooza-2025

Do SAEs only work for morphological traits, or do they discover functional traits?

Open
#4 1 comment 0 reactions 0 assignees View on GitHub
project idea question
Dominant language
No language data
Stars
5
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Like the title says. I expect SAEs to be very good at morphological traits because (as I understand it) they're very visual traits. But functional traits require a broader context. Does a vision model like DINOv2 understand functional traits? Can SAEs identify such traits? What about more biologically informed models like BioCLIP?

I think FishVista has functional trait annotations, so we could use that a test bed.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the FishVista functional trait annotations and the proposed DINOv2, BioCLIP, and SAE approaches. Design an evaluation that tests whether discovered features correspond to functional traits rather than only morphological ones. Done means reporting comparative results and documenting the methodology and conclusions.

Written by the indexing model from the issue text.

Assessment

Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.