cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams#

class cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams[source]#

Bases: object

Dataclass for parsing grad scaler params from optimizer params.

Methods

from_dict

Returns an instance of GradScalerParams from a dictionary.

Attributes

initial_loss_scale

loss_scaling_factor

max_gradient_norm

max_gradient_value

max_loss_scale

min_loss_scale

steps_per_increase

classmethod from_dict(params: Dict[str, Any]) → typing_extensions.Self[source]#

Returns an instance of GradScalerParams from a dictionary.

Note that matching keys are popped from the dictionary.

__init__(loss_scaling_factor: Optional[Union[float, str]] = 1.0, initial_loss_scale: Optional[float] = None, steps_per_increase: Optional[int] = 2000, min_loss_scale: Optional[float] = None, max_loss_scale: Optional[float] = None, max_gradient_norm: Optional[float] = None, max_gradient_value: Optional[float] = None) → None#

previous

cerebras.modelzoo.common.run_cstorch_flow.run_cstorch_train

next

cerebras.modelzoo.common.run_gpt_inference