huggingface / huggingface/candle

How to do freeze VarMap Vars?

Open
#2,323 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Hello everybody,

Is there away to freeze all Var Tensors in the VarMap like the below snippet ?
means something like implement the `Iterator` trait and detach the contained tensors from the graph and add a Var which can be trained !!!

```
# Freeze all the pre-trained layers
for param in model.parameters():
param.requires_grad = False

```

_Originally posted by @mohamed-180 in https://github.com/huggingface/candle/issues/891#issuecomment-2214407719_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the VarMap and Var APIs and checking how Var tensors participate in the computation graph; the issue names no files or tests. Define how freezing all existing variables should coexist with adding a trainable Var, then verify the behavior with a focused test for frozen pretrained layers and the new trainable variable.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.