cerebras.modelzoo.common.utils.model.lora.make_model_lora#

cerebras.modelzoo.common.utils.model.lora.make_model_lora(model: torch.nn.Module, lora_params_dict: Union[dict, List[dict]])[source]#

Create a Low Rank Adaptation (LoRA) model from a non-LoRA model. Note that the original non-LoRA model may be modified through this process.

Parameters

  • model – Initial model to make LoRA

  • lora_params_dict – LoRA parameters (in the form of a dict or list of dicts) which dictate how the supplied model will be converted into a LoRA model. The parameters should align with LoraConfig.

Returns

LoRA model

previous

cerebras.modelzoo.common.utils.model.lora.get_lora_config_for_module

next

cerebras.modelzoo.common.utils.model.lora.make_model_lora_helper