cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseConfigConverter_UnpackedHF_PackedCS#

class cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseConfigConverter_UnpackedHF_PackedCS[source]#

Bases: cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

Converter between a set of unpacked HF configs and a single packed CS configs.

Some CS models consist of separate components which we want to initialize from existing HF checkpoints. For example, initializing the image encoder and text decoder of a multimodal model. This converter class provides an abstraction for using existing HF <-> CS checkpoint converters.

In particular, we specify a list of BaseConfigConverter_HF_CS classes through converters() corresponding to each model component. Similarly, we specify another list of directory names through component_names() corresponding to the name of the subdirectory containing the model config.

During conversion, this converter applies the i-th component converter to the component config found in the i-th subdirectory name.

Methods

`assert_factory_fn`
`attempt_mup_to_sp`	Determines whether muP -> sP conversion is should be attempted.
`component_names`
`convert`
`convert_all_keys`
`convert_helper`	Converts all keys in a config from converter_indices.direction format to the other format.
`convert_key`	Attempts to convert the old key by matching against the list of conversion rules.
`converters`
`file_formats`
`formats`
`get_converter_indices`
`load`
`post_config_convert`
`pre_config_convert`
`replaceKey`	Copies value that exists at old_state_dict's old_key to new_state_dict's new_key.
`save`
`supports_conversion`
`supports_mup_conversion`	Determines whether muP -> sP conversion is supported for this model

convert_helper(config, converter_indices: cerebras.modelzoo.tools.checkpoint_converters.base_converter.FormatIndices, drop_unmatched_keys: bool = False, no_progress_bar: bool = True, debug: bool = False)[source]#: Converts all keys in a config from converter_indices.direction format to the other format. Conversion will fail if at least one of the keys did not match on any conversion rules and drop_unmatched_keys is not enabled. Returns the newly converted config.

__init__()#

attempt_mup_to_sp() → bool#: Determines whether muP -> sP conversion is should be attempted. Some HF models (such as BTLM) should not attempt muP -> sP conversion since they can natively handle muP.

convert_key(old_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, match_start: int = 0, prefix: str = '', action_fn_args: Optional[dict] = None, debug: bool = False) → None#: Attempts to convert the old key by matching against the list of conversion rules. The first rule to match is used for conversion (i.e. even if multiple rules would match, the latter ones are never used). Returns True if a conversion occurred.

static replaceKey(old_key: str, new_key: str, old_state_dict: collections.OrderedDict, new_state_dict: collections.OrderedDict, from_index: int, action_fn_args: Optional[dict] = None) → None#: Copies value that exists at old_state_dict’s old_key to new_state_dict’s new_key.

supports_mup_conversion() → bool#: Determines whether muP -> sP conversion is supported for this model

cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseConfigConverter_HF_CS

cerebras.modelzoo.tools.checkpoint_converters.base_converter.BaseDictionaryConverter