Unload lora specific lora weight

I've had trouble loading a few loras, and then unloading only a select few to free up more memory, but we only support unload_lora_weights() currently, which unloads all the loras - so I have to reload the loras I want again which is time consuming.

I would like a solution where we could specify the adapter_name associated with the lora_weight and unload that specific lora weight alone.