EleutherAI / EleutherAI/lm-evaluation-harness

Compatibility with Models from PyReft Library

Open
#2,012 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14k
Forks
3.6k
Avg merge
4d 2h
Merged PRs (30d)
35

Description

Hi everyone,

First, I'm sorry if this issue has already been raised.

I wanted to ask if the framework supports models obtained through the PyReft library (https://github.com/stanfordnlp/pyreft). Currently, in lm-eval, there is support for models obtained by applying LoRA through PEFT, but I haven’t found any information regarding loading models obtained via LoReft.

Is there anyone who can help me with this?

Thank you for your time and help!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.