Cerebras Wafer-Scale Cluster (R1.9.1)
Original Cerebras Installation Documentation (R1.6.1)
Cerebras AI Model Studio Documentation

Introduction

Overview
Weight Streaming Execution

Get Started

Run your first Cerebras job
Cerebras job scheduling and monitoring

Tutorials

Training and fine-tuning a Large Language Model (LLM)
Training a UNet for large image segmentation

How-To

Train an LLM using Maximal Update Parameterization
Fine-tune an LLM on a dataset using instructions
Train an LLM with a large or small context window
Train a model with weight sparsity (Beta)
Deterministically restart a dataloader
Control numerical precision level
Deploy Cerebras models
- Port a trained and fine-tuned model to Hugging Face
- Run Cerebras Model Zoo on a GPU

Prepare your data

Data Processing and Dataloaders
Preprocessing Scripts
Dataloaders and Processing scripts in Cerebras Model Zoo
Create your own dataloader
- Dataloaders for PyTorch
- Creating Custom Dataloaders for CS-2

Port your model

Model configuration using Cerebras Model Zoo
- Cerebras Model Zoo YAML parameters
- Convert checkpoints and configurations
  - Work with Cerebras checkpoints
Port your own architecture

Cerebras Features

Model Development
For training and evaluating any model
Boosting Model performance

Support

Troubleshooting
Software Release Notes
Documentation Updates
Glossary

Preprocessing Scripts#

Using Hugging Face datasets for auto-regressive LM
Creating HDF5 dataset for GPT models
Shuffling Samples for HDF5 dataset of GPT Models
Step by step guide to pre-process SlimPajama

previous

Data Processing and Dataloaders

next

Using Hugging Face datasets for auto-regressive LM

© Copyright 2023, Cerebras Systems.

Created using Sphinx 4.4.0.