facebookresearch / facebookresearch/SpinQuant

Integrate SpinQuant into Pruna AI

Open
#39 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
428
Forks
93
PR merge metrics
No merged PRs in 30d

Description

Hi @zxdmike @liuzechun 🤗

I'm Bertrand from Pruna AI. I came across your paper titled "SpinQuant LLM quantization with learned rotations" and was really impressed by your work! I believe it would be a great fit for the AI efficiency community, and I wanted to reach out to see if you'd be open to collaborating with us :)

We’ve developed the open-source [pruna](https://github.com/PrunaAI/pruna) package, which makes it easier for researchers and engineers to combine various compression methods in their work. Integrating your research could significantly enhance the package’s utility, enabling others to take advantage of your findings seamlessly.

Here’s what we’re hoping to do:

- Integrate your research: We’d love to bring your methods into pruna to make them easily accessible to the community.

- Showcase your work: This will allow others to find your research through our project, and you can also link to any models or datasets that are relevant.

- Enhance visibility: Your research will be visible to a broader community of AI engineers and researchers. We can especially include it in AI efficiency [materials](https://github.com/PrunaAI/awesome-ai-efficiency), [courses](https://github.com/PrunaAI/ai-efficiency-courses), [tutorials](https://docs.pruna.ai/en/stable/docs_pruna/tutorials/index.html#), and [blogs](https://www.pruna.ai/blog) :)

If you're open to this, here are the detailed [contributions guidelines](https://docs.pruna.ai/en/stable/docs_pruna/contributions/how_to_contribute.html#setup) with steps to integrate a new [algorithm](https://docs.pruna.ai/en/stable/docs_pruna/user_manual/customize_algorithm.html), [dataset](https://docs.pruna.ai/en/stable/docs_pruna/user_manual/customize_dataset.html), or [metric](https://docs.pruna.ai/en/stable/docs_pruna/user_manual/customize_metric.html). The end goal would be that anyone can compress and evaluate their model with your research with:

# Apply Compression Algorithm to ML Models
```
model = ...

smash_config = SmashConfig()
smash_config["algo_group_1"] = "algo_1"
smash_config["algo_group_2"] = "algo_2"
smashed_model = smash(model=model, smash_config=smash_config)
```

# Evaluate ML Models
```
metrics = ["latency", "disk_memory", "energy_consumed", ...]
datamodule = PrunaDataModule(datasets=["dataset1", "dataset2"], collate_fn="image_generation_collate")

task = Task(metrics, datamodule=datamodule)
eval_agent = EvaluationAgent(task)
model_results = eval_agent.evaluate(smashed_model)
```

Let me know if you're interested/need any help regarding this!

Have a great day,
Bertrand - Cofounder, President, Chief Scientist @ Pruna AI
[Twitter](https://twitter.com/Bertrand_Charp)|[LinkedIn](https://www.linkedin.com/in/bertrand-charpentier-76995ab6/)|[Scholar](https://scholar.google.com/citations?user=0rqI-ycAAAAJ&hl=de)|[Website](https://sharpenb.github.io/)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.