cerebras.modelzoo.data_preparation.nlp.transformer.split_files

Split input src/tgt file into multiple files. Number of examples per file is controlled by buffer_len param.

Functions

main

previous

cerebras.modelzoo.data_preparation.nlp.transformer.create_meta.main

next

cerebras.modelzoo.data_preparation.nlp.transformer.split_files.main