cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5Dataset

class cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5Dataset(*args, **kwargs)

Bases: cerebras.modelzoo.data.common.h5_map_dataset.dataset.HDF5Dataset

Dataset class to handle image preprocessing in multimodal datasets.

This class is largely the same as the parent class HDF5Dataset except with added image loading and preprocessing.

Parameters

config (cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5DatasetConfig) – The config used to configure the dataset.

Methods

generate_sample
load_state_dict
map
preprocess_img
state_dict

Attributes

by_sample

previous

cerebras.modelzoo.data.common.h5_map_dataset.dataset.MLMHDF5Dataset

next

cerebras.modelzoo.data.common.h5_map_dataset.dataset.MultiModalHDF5DatasetConfig