cerebras.modelzoo.data_preparation.utils.get_files_in_metadata#

cerebras.modelzoo.data_preparation.utils.get_files_in_metadata(metadata_filepaths)[source]#

Function to read the files in metadata file provided as input to data generation scripts.

Parameters

metadata_filepaths – path/s to metadata files

Returns List input_files

Contents of metadata files.

previous

cerebras.modelzoo.data_preparation.utils.create_masked_lm_predictions

next

cerebras.modelzoo.data_preparation.utils.get_label_id_map