cerebras.modelzoo.data.common.input_utils.shard_list_interleaved#

cerebras.modelzoo.data.common.input_utils.shard_list_interleaved(input_list, worker_id, num_workers)[source]#

Shards a list by assigning consecutive elements to alternating workers (i.e. interleaving). If the length of the list is not divisible by the number of workers, the remainder elements are spread across a subset of the workers such that each worker in the subset receives 1 extra element.

Parameters

  • input_list (list) – list to shard in an interleaved fashion

  • worker_id (int) – index of shard to return

  • num_workers (int) – number of shards to create

Returns

worker_id’s shard (a subset of input_list).

previous

cerebras.modelzoo.data.common.input_utils.shard_list_contiguous

next

cerebras.modelzoo.data.common.input_utils.shard_list_of_chunks_contiguous