cerebras.modelzoo.common.utils.model.lora.LoRA_Embedding#
- class cerebras.modelzoo.common.utils.model.lora.LoRA_Embedding[source]#
Bases:
torch.nn.Embedding
,cerebras.modelzoo.common.utils.model.lora.LoRALayer
LoRA embedding layer From https://github.com/microsoft/LoRA/blob/main/loralib/layers.py.
Methods
forward
reset_parameters
train
- __init__(num_embeddings: int, embedding_dim: int, r: int = 0, lora_alpha: int = 1, merge_weights: bool = True, **kwargs)[source]#
- __call__(*args: Any, **kwargs: Any) Any #
Call self as a function.
- static __new__(cls, *args: Any, **kwargs: Any) Any #