bigscience-workshop / bigscience-workshop/petals
How to specify lora parameters
Open
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 650
- PR merge metrics
- No merged PRs in 30d
Description
When running an entire bloom model in local environment, I can view the information of all layers and specify the query_key_value module in lora. But in petals, the (h) layer becomes a remote sequential. How should I specify the target module in lora like this:
```
config = LoraConfig(
r=16,
lora_alpha=16,
target_modules=["query_key_value"],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
```
## print bloom in device:
## print bloom in petals:
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.