Script that normalizes text
Functions
normalize_text
parse_args
recreate_dataset
previous
cerebras.modelzoo.data_preparation.nlp.slimpajama.preprocessing.filter.get_short_documents
next
cerebras.modelzoo.data_preparation.nlp.slimpajama.preprocessing.normalize_text.normalize_text