cerebras.modelzoo.data_preparation.huggingface.HuggingFaceDataProcessor#

Pytorch HuggingFace Dataloader

Classes

HuggingFaceDataProcessor

A HuggingFace map-style Data Processor. :param dict params: dict containing training input parameters for creating dataset. Expects the following fields:.

HuggingFaceDataProcessorConfig

previous

cerebras.modelzoo.data_preparation.huggingface.HF_converter_example_Eli5.main

next

cerebras.modelzoo.data_preparation.huggingface.HuggingFaceDataProcessor.HuggingFaceDataProcessor