cerebras.modelzoo.data_preparation.nlp.slimpajama.utils

Functions

cycle_documents
rm_if_exists
sha256str
utf8len
write_lmd_dataset

previous

cerebras.modelzoo.data_preparation.nlp.slimpajama.split_dataset.write_samples

next

cerebras.modelzoo.data_preparation.nlp.slimpajama.utils.cycle_documents