cerebras.modelzoo.data_preparation.raw_dataset_processor.RawDatasetProcessor#
This is Dataset process for processing Raw data set on the fly This contains methods for loading the dataset, tokenizing the dataset and all data transformations are handled as part of the collator function
Classes
Dataset processor for multimodal data (e.g., image data). |
|
Multimodal Configuration class for RawDatasetProcessor. |
|
Configuration class for RawDatasetProcessor. |