logo

Software Documentation (Version 1.6.1)

  • Software Release Notes
  • Documentation Updates

Cerebras Basics

  • How Cerebras Works
  • The Cerebras ML Workflow
  • Cerebras Execution Modes

Getting Started

  • Software Requirements and Dependencies
  • Checklist Before You Quickstart
  • TensorFlow: Quick-Start Guide
    • Weight Streaming Appliance Workflow
    • Pipeline K8s Workflow
    • Pipeline Slurm/Singularity Workflow
  • Pytorch: Quick-Start Guide
    • Weight Streaming Appliance Mode Workflow
    • Pipeline K8s Workflow
    • Pipeline Slurm/Singularity Workflow

Reference Samples

  • Cerebras Model Zoo

General

  • DataLoader Overview
    • PyTorch DataLoader
    • Basics of TFRecords
  • Data Formats
  • Dynamic Loss Scaling
  • Performance Optimization Practices
  • Multi-Replica Data Parallel Training

Scripts and Templates

  • The run.py Template
  • The csrun_cpu Script
  • The csrun_wse Script
  • The cs_input_analyzer Script

Develop with TensorFlow

  • Workflow for TensorFlow on CS
  • Port TensorFlow to Cerebras
    • Keras Model to CerebrasEstimator
    • Using the CerebrasEstimator
    • The CerebrasEstimator Interface
    • TensorFlow Estimator to CerebrasEstimator
    • Limitations of the CerebrasEstimator
  • Prepare Input
    • Multi-Worker Input Pipeline
    • Sharding For the CS system
    • The CS_AUTOTUNE
    • Optimize Input Function
  • TensorFlow Dynamic Loss Scaling
  • Compile on CPU
  • Train, Eval, and Predict
  • Early Stopping
  • Multiple Models
    • Multi-model Inference
  • TensorFlow Variable Sequence Length
  • Using TensorBoard
  • Best Practices
  • Supported TensorFlow Layers
    • tf package
      • tf.layers.ActivationLayer module
      • tf.layers.AddLayer module
      • tf.layers.AttentionLayer module
      • tf.layers.BaseLayer module
      • tf.layers.Conv2DLayer module
      • tf.layers.Conv2DTransposeLayer module
      • tf.layers.CrossEntropyFromLogitsLayer module
      • tf.layers.DenseLayer module
      • tf.layers.DropoutLayer module
      • tf.layers.EmbeddingLayer module
      • tf.layers.FeedForwardNetwork module
      • tf.layers.FeedForwardNetworkV2 module
      • tf.layers.Input module
      • tf.layers.LayerNormalizationLayer module
      • tf.layers.MaxPool2DLayer module
      • tf.layers.PoolerLayer module
      • tf.layers.PoolerLayerV2 module
      • tf.layers.PositionEmbeddingLayer module
      • tf.layers.PrePostProcessWrapper module
      • tf.layers.ReshapeLayer module
      • tf.layers.SegmentEmbeddingLayer module
      • tf.layers.SharedWeightsDenseLayer module
      • tf.layers.SoftmaxLayer module
      • tf.layers.SquaredErrorLayer module

Develop with PyTorch

  • Workflow for PyTorch on CS
  • Porting PyTorch Model to CS
  • PyTorch Runners
  • PyTorch Variable Tensor Shape
  • Limitations of PyTorch on Cerebras
  • Cerebras PyTorch Layer API
    • modelzoo.common.pytorch.layers.MultiheadAttention
    • modelzoo.common.pytorch.layers.TransformerDecoderLayer
    • modelzoo.common.pytorch.layers.TransformerDecoder
    • modelzoo.common.pytorch.layers.TransformerEncoderLayer
    • modelzoo.common.pytorch.layers.TransformerEncoder

Compiler Reports

  • Input Function Report
  • Compile Report
  • Incremental Compile

Extensions

  • Adding Custom Packages To cbcore Container
Theme by the Executable Book Project
  • .rst

Port TensorFlow to Cerebras

Port TensorFlow to Cerebras#

This section describes how to port your TensorFlow code to Cerebras.

  • Keras Model to CerebrasEstimator
    • Using the KerasModelToCerebrasEstimator
  • Using the CerebrasEstimator
    • Calling the CerebrasEstimator
    • Callback input function
    • Callback model function
    • Setting the runtime configuration
  • The CerebrasEstimator Interface
    • Syntax
    • Arguments
    • Methods
  • TensorFlow Estimator to CerebrasEstimator
    • Step 1: Model function
    • Step 2: Input function
    • Step 3: Use CerebrasEstimator
    • Step 4: Edit RunConfig
    • Step 5: Ensure mixed precision
  • Limitations of the CerebrasEstimator
    • Model function limitations
    • Input function differences
    • Input function limitations
    • Config differences
    • Compilation differences

previous

Workflow for TensorFlow on CS

next

Keras Model to CerebrasEstimator

© Copyright 2022, Cerebras Systems.
Last updated on Jan 30, 2024, 9:20:44 PM.