cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams

class cerebras.modelzoo.common.run_cstorch_flow.GradScalerParams(loss_scaling_factor=1.0, initial_loss_scale=None, steps_per_increase=2000, min_loss_scale=None, max_loss_scale=None, max_gradient_norm=None, max_gradient_value=None)

Bases: object

Dataclass for parsing grad scaler params from optimizer params.

Methods

from_dict

Returns an instance of GradScalerParams from a dictionary.

Attributes

initial_loss_scale
loss_scaling_factor
max_gradient_norm
max_gradient_value
max_loss_scale
min_loss_scale
steps_per_increase

classmethod from_dict(params)

Returns an instance of GradScalerParams from a dictionary.

Note that matching keys are popped from the dictionary.

previous

cerebras.modelzoo.common.run_cstorch_flow.setup_hf_env_vars

next

cerebras.modelzoo.common.run_eleuther_eval_harness