facebookresearch / facebookresearch/large_concept_model

Inquiry: Have recursive or other Transformer variants been explored besides BaseLCM and Diffusion models?

Open
#37 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.4k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

Hi LCM team,

I've been exploring the large_concept_model repository and noticed the implementations of BaseLCM, Single Tower Diffusion, and Two Tower Diffusion models. I am curious if the team has experimented with or considered other Transformer variants, such as recursive transformers or their recent evolutions like Relaxed Recursive Transformers or Mixture-of-Recursions models?

Given the efficiency and parameter-sharing advantages of recursive transformers researched in recent papers, I wonder if they could be beneficial or are planned for this codebase.

Would appreciate any insights or pointers regarding this!

Thank you for your work on this great project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.