cerebras.modelzoo.data.nlp.gpt.HuggingFaceIterableDataProcessorEli5.HuggingFaceIterableDataProcessorEli5#

class cerebras.modelzoo.data.nlp.gpt.HuggingFaceIterableDataProcessorEli5.HuggingFaceIterableDataProcessorEli5(config)[source]#

Bases: cerebras.modelzoo.data_preparation.huggingface.HuggingFaceDataProcessor.HuggingFaceDataProcessor

A HuggingFace Eli5 Iterable Data Processor.

Parameters

config (cerebras.modelzoo.data.nlp.gpt.HuggingFaceIterableDataProcessorEli5.HuggingFaceIterableDataProcessorEli5Config) – The configuration object

Methods

create_dataloader

Classmethod to create the dataloader object.
create_dataloader()#

Classmethod to create the dataloader object.

previous

cerebras.modelzoo.data.nlp.gpt.HuggingFaceIterableDataProcessorEli5

next

cerebras.modelzoo.data.nlp.gpt.HuggingFaceIterableDataProcessorEli5.HuggingFaceIterableDataProcessorEli5Config