Cerebras Wafer-Scale Cluster (R1.9.1)

Get started

Get started by preparing and setting up your Cerebras environment to get your first Cerebras job up and running. Follow our Get Started guides:

• 1. Set up Cerebras's environment
• 2. Clone Cerebras Model Zoo
• 3. Launch your first job

Tutorials

Acquaint yourself with using Cerebras's Wafer-Scale to preprocess data, load data, train, and fine-tune deep neural networks using our step-by-step instructional tutorials:

• Train and fine-tune a Large Language Model (LLM)
• Train a UNet model for large image segmentation

How-to

Find how-to guides to use Cerebras-enhanced features:

• Train a GPT model using Maximum Update Parametrization
• Fine-tune an LLM on a dataset using instructions
• Train a model with a large or small context window
• Train a model with weight sparsity
• Deterministically start a dataloader
• Control numerical precision level
• Deploy Cerebras models

Port your model

If your model is one of the supported models in Cerebras Model Zoo, you will port your model by:

• Preprocessing your data using Cerebras Model Zoo, including Hugging Face datasets
• Creating your own data processors
• Modifying configuration files
• Converting configurations and checkpoints from Hugging Face

If you are developing a new model, you will use the layers supported in the Cerebras Layer API and one of these approaches:

• Port your model using run function from Cerebras Model Zoo
• (Early access) Create your own run function using Cerebras PyTorch API. This will be the default approach starting with R2.0.

Cerebras Features

When developing your model:

• Understand kernel autogeneration with Autogen
• Define environment variables for input workers
• Import user-specific dependencies to Cerebras environments
• Follow best practices for CV dataloaders

When training and evaluating your model:

• Evaluate your model during training
• Summarize scalars and tensors
• Use dynamic loss scaling

To boost the performance of your model:

• Measure throughput of your model
• Train with number of tokens loss scaling
• Train with gradient accumulation

 

previous

Explore Cerebras’ Documentation

next

Overview