cerebras.modelzoo.config_manager.config_classes.base.data_config.DataProcessorConfig#
- class cerebras.modelzoo.config_manager.config_classes.base.data_config.DataProcessorConfig(batch_size: int = <object object at 0x7f9345f8db90>, shuffle: bool = True, shuffle_seed: int = 0, num_workers: int = 0, prefetch_factor: Optional[int] = None, persistent_workers: Optional[bool] = None)[source]#
- batch_size: int = <object object>#
Batch size to be used
- shuffle: bool = True#
Whether or not to shuffle the dataset
- shuffle_seed: int = 0#
Seed used for deterministic shuffling
- num_workers: int = 0#
The number of PyTorch processes used in the dataloader
- prefetch_factor: Optional[int] = None#
The number of batches to prefetch in the dataloader
- persistent_workers: Optional[bool] = None#
Whether or not to keep workers persistent between epochs