cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.create_hdf5_dataset

Script to generate an HDF5 dataset for GPT Models.

Functions

main	Main function for execution.

previous

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.chunk_data_preprocessor.ChunkDataPreprocessor

next

cerebras.modelzoo.data_preparation.nlp.chunk_data_processing.create_hdf5_dataset.main