lightly-ai / lightly-ai/lightly

New SSL method: Franca (Nested Matryoshka Clustering)

Open
#2,010 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.8k
Forks
367
Avg merge
3d 22h
Merged PRs (30d)
5

Description

## Overview

Franca (Venkataramanan et al., valeoai) looks like a good fit for lightly's DINO/DINOv2 family.

- **Paper:** https://arxiv.org/abs/2507.14137
- **Code:** https://github.com/valeoai/Franca (Apache-2.0)

## Franca contributions

It's a fully open DINOv2-based model (code, data, weights). It keeps the DINOv2 setup and adds:

- **Nested "Matryoshka" clustering**: the clustering head runs at several nested embedding sizes at once, giving coarse-to-fine assignments.
- **Cyclic-shift masking** for the iBOT branch.
- A post-hoc **positional-disentanglement head (RASA)**.

On ImageNet-21K, and separately a LAION subset, it reports matching or surpassing DINOv2, CLIP, and SigLIPv2.

## Fit with lightly

Most of it is the DINOv2 recipe already in lightly, so the main new pieces would be the Matryoshka head and loss, plus the cyclic-shift masking. I think we should leave out RASA as it's post-hoc (?)

Would it be of interest to include the method in lightly? If so I can try to implement it after agreeing on the implementation plan.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing lightly's existing DINO/DINOv2 implementation, then compare it with the Franca paper and the linked valeoai/Franca code. Clarify the implementation plan for the nested Matryoshka clustering head and loss and cyclic-shift masking, including whether RASA is excluded; done means the scope and design are agreed before coding.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.