cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.add_lm_args#

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.add_lm_args(parser)[source]#

The language-modeling format is common enough (FIM is very similar) that we can re-use the arguments for it

previous

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.add_llava_phase_2_args

next

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.utils.add_mlm_args