cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_sP_muP#

class cerebras.modelzoo.tools.checkpoint_converters.mup.Converter_sP_muP[source]#

Transforms a CS muP checkpoints into a CS sP checkpoint.

muP: Maximal Update Parametrization. sP: Standard Parametrization.

Methods

`convert_all_keys`
`convert_key`	Attempts to convert the old key by matching against the list of conversion rules.
`formats`
`get_converter_indices`
`is_mup`
`replaceKey`	Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.
`scale_embedding_layernorm`
`scale_embeddings`
`scale_k_projection`
`scale_lm_head`
`supports_conversion`

convert_key(old_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, match_start: int = 0, prefix: str = '', action_fn_args: Optional[dict] = None, debug: bool = False) → None#: Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.

static replaceKey(old_key: str, new_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, action_fn_args: Optional[dict] = None) → None#: Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.

cerebras.modelzoo.tools.checkpoint_converters.mup.ConfigConverter_sP_muP

cerebras.modelzoo.tools.checkpoint_converters.qa