cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.create_hdf5_dataset

Script that generates a dataset in HDF5 format for GPT Models.

Functions

main	Main function for execution.

previous

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.convert_dataset_to_HDF5.write_hdf5_file

next

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.create_hdf5_dataset.main