logo

Software Documentation (Version 1.6.1)
  • Software Release Notes
  • Documentation Updates

Cerebras Basics

  • How Cerebras Works
  • The Cerebras ML Workflow
  • Cerebras Execution Modes

Getting Started

  • Software Requirements and Dependencies
  • Checklist Before You Quickstart
  • TensorFlow: Quick-Start Guide
    • Weight Streaming Appliance Workflow
    • Pipeline K8s Workflow
    • Pipeline Slurm/Singularity Workflow
  • Pytorch: Quick-Start Guide
    • Weight Streaming Appliance Mode Workflow
    • Pipeline K8s Workflow
    • Pipeline Slurm/Singularity Workflow

Reference Samples

  • Cerebras Model Zoo

General

  • DataLoader Overview
    • PyTorch DataLoader
    • Basics of TFRecords
  • Data Formats
  • Dynamic Loss Scaling
  • Performance Optimization Practices
  • Multi-Replica Data Parallel Training

Scripts and Templates

  • The run.py Template
  • The csrun_cpu Script
  • The csrun_wse Script
  • The cs_input_analyzer Script

Develop with TensorFlow

  • Workflow for TensorFlow on CS
  • Port TensorFlow to Cerebras
    • Keras Model to CerebrasEstimator
    • Using the CerebrasEstimator
    • The CerebrasEstimator Interface
    • TensorFlow Estimator to CerebrasEstimator
    • Limitations of the CerebrasEstimator
  • Prepare Input
    • Multi-Worker Input Pipeline
    • Sharding For the CS system
    • The CS_AUTOTUNE
    • Optimize Input Function
  • TensorFlow Dynamic Loss Scaling
  • Compile on CPU
  • Train, Eval, and Predict
  • Early Stopping
  • Multiple Models
    • Multi-model Inference
  • TensorFlow Variable Sequence Length
  • Using TensorBoard
  • Best Practices
  • Supported TensorFlow Layers
    • tf package
      • tf.layers.ActivationLayer module
      • tf.layers.AddLayer module
      • tf.layers.AttentionLayer module
      • tf.layers.BaseLayer module
      • tf.layers.Conv2DLayer module
      • tf.layers.Conv2DTransposeLayer module
      • tf.layers.CrossEntropyFromLogitsLayer module
      • tf.layers.DenseLayer module
      • tf.layers.DropoutLayer module
      • tf.layers.EmbeddingLayer module
      • tf.layers.FeedForwardNetwork module
      • tf.layers.FeedForwardNetworkV2 module
      • tf.layers.Input module
      • tf.layers.LayerNormalizationLayer module
      • tf.layers.MaxPool2DLayer module
      • tf.layers.PoolerLayer module
      • tf.layers.PoolerLayerV2 module
      • tf.layers.PositionEmbeddingLayer module
      • tf.layers.PrePostProcessWrapper module
      • tf.layers.ReshapeLayer module
      • tf.layers.SegmentEmbeddingLayer module
      • tf.layers.SharedWeightsDenseLayer module
      • tf.layers.SoftmaxLayer module
      • tf.layers.SquaredErrorLayer module

Develop with PyTorch

  • Workflow for PyTorch on CS
  • Porting PyTorch Model to CS
  • PyTorch Runners
  • PyTorch Variable Tensor Shape
  • Limitations of PyTorch on Cerebras
  • Cerebras PyTorch Layer API
    • modelzoo.common.pytorch.layers.MultiheadAttention
    • modelzoo.common.pytorch.layers.TransformerDecoderLayer
    • modelzoo.common.pytorch.layers.TransformerDecoder
    • modelzoo.common.pytorch.layers.TransformerEncoderLayer
    • modelzoo.common.pytorch.layers.TransformerEncoder

Compiler Reports

  • Input Function Report
  • Compile Report
  • Incremental Compile

Extensions

  • Adding Custom Packages To cbcore Container
Theme by the Executable Book Project
  • .rst

Prepare Input

Prepare Input#

  • Multi-Worker Input Pipeline
    • Dataset for input workers
    • Configuring multi-worker input pipeline
    • Determinism
    • Shuffling buffers
    • Optimizing Input Pipeline
  • Sharding For the CS system
    • Data in tf.data.Dataset
    • Data in files
    • Comparison
  • The CS_AUTOTUNE
    • Using CS_AUTOTUNE
  • Optimize Input Function
    • Automatic execution of the analyzer

previous

Limitations of the CerebrasEstimator

next

Multi-Worker Input Pipeline

© Copyright 2022, Cerebras Systems.
Last updated on Jan 30, 2024, 9:20:44 PM.