cerebras.modelzoo.data_preparation.data_preprocessing.utils.setup_warning_logging

cerebras.modelzoo.data_preparation.data_preprocessing.utils.setup_warning_logging(output_dir, module_name)

Set up logging to log warnings to a file in the specified output directory.

Parameters

output_dir (str) – The directory where the warnings log file should be stored.

previous

cerebras.modelzoo.data_preparation.data_preprocessing.utils.pad_helper

next

cerebras.modelzoo.data_preparation.data_preprocessing.utils.split_text_and_tokenize