bigscience-workshop / bigscience-workshop/petals

How to specify lora parameters

Open
#322 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
650
PR merge metrics
No merged PRs in 30d

Description

When running an entire bloom model in local environment, I can view the information of all layers and specify the query_key_value module in lora. But in petals, the (h) layer becomes a remote sequential. How should I specify the target module in lora like this:

```
config = LoraConfig(
r=16,
lora_alpha=16,
target_modules=["query_key_value"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
```

## print bloom in device:

## print bloom in petals:

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.