cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultimodalSimpleHDF5Dataset#

class cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultimodalSimpleHDF5Dataset(*args, **kwargs)[source]#

Bases: cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5Dataset

Dataset class to handle image preprocessing in multimodal datasets.

This class is largely the same as the parent class MultimodalHDF5Dataset except with added support for multiple images and intermingling of text and images.

Parameters

config (cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultimodalSimpleHDF5DatasetConfig) – The config used to configure the dataset.

Methods

generate_sample

load_state_dict

map

preprocess_img

state_dict

Attributes

by_sample

previous

cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5DatasetConfig

next

cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultimodalSimpleHDF5DatasetConfig