Unfreeze layers after few epochs : Runtime Error

Hello,

I am trying to train a model unfreezing some layers after few epochs without stopping and restarting the training.
For instance:

# unfreeze layers
def unfreeze(model):  
    for params in model.parameters():
        params.requires_grad = True

#unfreeze layers and return new optimizer
def update_layers_and_optimizer(model, args,  n=9):
    unfreeze(model)
    if n > 0:
        freeze_resnet(list(model.children())[0][:n])

    optim_params = list(filter(lambda p: p.requires_grad, model.parameters()))
    print (' Update : Optimizing %d paramters'%len(optim_params))

        optimizer = optim.SGD(optim_params, lr=args.lr, weight_decay=args.weight_decay, momentum=args.momentum, nesterov=True)
    return optimizer
####
put code to call the model and data ...
####

for e in range(epochs):
   ## compute loss etc
   
   if e == 5:
       optimizer = update_layers_and_optimizer(net,  n=7)
       net, optimizer = amp.initialize(net, optimizer, opt_level="O2", 
                                            keep_batchnorm_fp32=True, loss_scale="dynamic")

Unfortunately I got an runtime error :

RuntimeError: Found param inception.0.conv.weight with type torch.cuda.HalfTensor, expected torch.cuda.FloatTensor.
When using amp.initialize, you do not need to call .half() on your model
before passing it, no matter what optimization level you choose.

I was wondering how to avoid this error and if there is a way to unfreeze layers without stopping the training ?

Thanks for the help

No repository files or tests are named. Start by reproducing the shown epoch loop with Apex AMP, especially the second amp.initialize call after replacing the optimizer, and trace the parameter dtypes and AMP state. Done means determining whether the requested mid-training unfreezing can work without the reported dtype error, or documenting the supported limitation.