CarperAI / CarperAI/trlx

Add support for safetensors

Open
#505 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
Python
Stars
4.8k
Forks
487
PR merge metrics
No merged PRs in 30d

Description

### 🚀 The feature, motivation, and pitch

By default, PyTorch uses the Pickle format to save and load models. Unloading a Pickle file could unfortunately lead to remote code execution, which is a grave vulnerability. So any Pickle model that you load must come from a trusted source. Safetensors addresses that. It uses a different file format, and defines alternative functions to torch.load and torch.save. Using safetensors shouldn't impact the inference, only the loading and saving. It also offers a few other advantages such as "zero-copy" or lazy loading, which they say can significantly improve the loading time especially on distributed settings.

Note : support for safetensors is planned in peft 0.4.0

### Alternatives

_No response_

### Additional context

screenshots from https://github.com/huggingface/safetensors :
![image](https://github.com/CarperAI/trlx/assets/47578089/4597896e-95b8-4195-a31d-a38c26f8e6bb)
![image](https://github.com/CarperAI/trlx/assets/47578089/58a799f7-a25e-4eaf-b26e-5cf72928b9d0)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.