.. _pytorch-cs-workflow:

Workflow for PyTorch on CS
==========================

When you are targeting the Cerebras system for your neural network jobs, start with the high-level workflow described here.

.. raw:: html

    <div class="container-fluid">

    <div class="row">
        <div class="col">
            <div class="card h-100" >
                <div class="card-body">
                    <h3 class="card-title">1</h3>
                    <a href="#port-pytorch-to-cs" class="card-link"><h4>Port PyTorch to CS</h4></a>
                </div>
            </div>
        </div>
        <div class="col">
            <div class="card h-100" >
                <div class="card-body">
                    <h3 class="card-title">2</h3>
                    <a href="#load-the-data" class="card-link"><h4>Load the data</h4></a>
                </div>
            </div>
        </div>
        <div class="col">
            <div class="card h-100" >
                <div class="card-body">
                    <h3 class="card-title">3</h3>
                    <a href="#compile-on-cpu" class="card-link"><h4>Compile on CPU</h4></a>
                </div>
            </div>
        </div>

        <div class="col">
            <div class="card h-100" >
                <div class="card-body">
                    <h3 class="card-title">4</h3>
                    <a href="#train-on-the-cs-system" class="card-link"><h4>Train on CS</h4></a>
                </div>
            </div>
        </div>

        <!-- break -->

    </div>
    </div>
    <br \>


.. attention::

	Several PyTorch models, such as the following, and run scripts, are provided in `Cerebras Model Zoo <https://github.com/Cerebras/modelzoo>`_:

        - `PyTorch version of FC-MNIST <https://github.com/Cerebras/modelzoo/tree/main/modelzoo/fc_mnist/pytorch>`_, and

        - `PyTorch versions of BERT Base and BERT Large <https://github.com/Cerebras/modelzoo/tree/main/modelzoo/transformers/pytorch/bert>`_.


1. Port PyTorch to CS
---------------------

See :ref:`porting-pytorch-to-cs` for detailed documentation on how to port your
PyTorch model to work on a Cerebras System.


2. Load the data
----------------

As mentioned in :ref:`porting-pytorch-to-cs`, you need to provide
:code:`get_train_dataloader` for training and :code:`get_eval_dataloader` for
evaluation. See :ref:`cbtorch-dataloader` for details.

3. Compile on CPU
-----------------

Before you run it on the Cerebras system, we recommend that you iterate until
your model first compiles successfully on a CPU node. Make sure that this CPU
node has the Cerebras Singularity container client software.

As described in :ref:`porting-pytorch-to-cs`, you can verify that your model
will compile by passing in the :code:`--compile_only` flag to the :code:`run`
function's CLI.

4. Train on the Cerebras system
-------------------------------

As described in :ref:`porting-pytorch-to-cs`, you can train or evaluate your
model by providing the IP address of the Cerebras system and the mode you want
to run.