Distributed Methods Intra-Layer Parallel Model Parallel Methods GShard public – 2 min read Have you ever been frustrated by slow or inefficient neural network computations? If so, you may be interested in GShard,… Apr 23, 2023 Devin Schumacher
Distributed Methods Intra-Layer Parallel Model Parallel Methods Mesh-TensorFlow public – 2 min read Overview of Mesh-TensorFlow Mesh-TensorFlow is a programming language used to distribute tensor computations. Like data-parallelism that splits tensors and operations… Apr 23, 2023 Devin Schumacher
Distributed Methods Intra-Layer Parallel Model Parallel Methods Tofu public – 2 min read Overview of Tofu Tofu is a system designed to partition large deep neural network (DNN) models across multiple GPU devices,… Apr 23, 2023 Devin Schumacher