性能结果
在这里，我们提供了我们最近提交的MLPerf训练1.1版本的结果，以及我们自己在针对训练和推理的更广泛模型中进行的基准测试活动的结果。
MLPerf训练1.1版本表现
对于我们向MLPerf训练1.1版本的提交，我们选择了提交流行应用程序基准类别，包括图像分类（ResNet-50）和自然语言处理（BERT)。
提交有两个分区（Division)。封闭分区（Closed Division）要求提交者使用完全相同的模型和优化器实施，包括定义超参数状态和训练时期。还有一个开放分区（Open Division)，通过支持更适合不同处理器功能的不同模型实现来促进和支持创新，但确保达到与封闭分区完全相同的模型准确性和质量。
| Division | Model | MLPerf Quality Target | Platform | SDK Version | Framework | MLPerf ID | Dataset | Precision | Time to Train (mins) |
|---|---|---|---|---|---|---|---|---|---|
| Closed | ResNet50 v1.5 | 75.90% classification | IPU-POD16 | SDK 2.3.0 | TensorFlow | 1.1-2040 | ImageNet2012 | 16.16 | 28.33 |
| Closed | ResNet50 v1.5 | 75.90% classification | IPU-POD64 | SDK 2.3.0 | TensorFlow | 1.1-2042 | ImageNet2012 | 16.16 | 8.50 |
| Closed | ResNet50 v1.5 | 75.90% classification | IPU-POD128 | SDK 2.3.0 | TensorFlow | 1.1-2044 | ImageNet2012 | 16.16 | 5.67 |
| Closed | ResNet50 v1.5 | 75.90% classification | IPU-POD256 | SDK 2.3.0 | TensorFlow | 1.1-2045 | ImageNet2012 | 16.16 | 3.79 |
| Closed | BERT | 0.72 Mask-LM accuracy | IPU-POD16 | SDK 2.3.0 | PopART | 1.1-2039 | Wikipedia | 16.16 | 32.70 |
| Closed | BERT | 0.72 Mask-LM accuracy | IPU-POD64 | SDK 2.3.0 | PopART | 1.1-2041 | Wikipedia | 16.16 | 10.56 |
| Closed | BERT | 0.72 Mask-LM accuracy | IPU-POD128 | SDK 2.3.0 | PopART | 1.1-2043 | Wikipedia | 16.16 | 6.86 |
| Open | BERT | 0.72 Mask-LM accuracy | IPU-POD16 | SDK 2.3.0 | PopART | 1.1-2088 | Wikipedia | 16.16 | 26.05 |
| Open | BERT | 0.72 Mask-LM accuracy | IPU-POD64 | SDK 2.3.0 | PopART | 1.1-2089 | Wikipedia | 16.16 | 8.25 |
| Open | BERT | 0.72 Mask-LM accuracy | IPU-POD128 | SDK 2.3.0 | PopART | 1.1-2087 | Wikipedia | 16.16 | 5.88 |
MLPerf的名称和徽标是MLCommons联盟（MLCommons Association）在美国和其他国家的商标。
版权所有，严禁未经授权使用。有关更多信息，请访问 www.mlperf.org
训练：吞吐量
训练机器学习模型涉及在输入数据集（训练数据）上运行算法，直到模型收敛，收敛意味着它已经学会以指定的准确性产生所需的输出。在此语境中，吞吐量被定义为模型每秒处理的输入数据点（序列、图像或行）的数量。吞吐量通常用作衡量硬件性能的指标，因为它与将模型训练达到指定准确性的时间直接相关。
下面提供的结果详细说明了在指定配置中每个参考模型获得的吞吐量值。在真实数据上运行的所有配置都针对收敛进行了验证。
| Model | Variant | Platform | SDK Version | Framework | Dataset | Batch Size | Precision | Throughput (items/sec) |
|---|---|---|---|---|---|---|---|---|
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD16 | SDK 2.3.0 | PopART | Wikipedia | 65,536 | 16.16 | 3713 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD16 | SDK 2.3.0 | TensorFlow | Wikipedia | 65,600 | 16.16 | 3664 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD16 | SDK 2.3.0 | PyTorch | Wikipedia | 65,536 | 16.16 | 3562 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD64 | SDK 2.3.0 | PopART | Wikipedia | 65,536 | 16.16 | 14122 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD64 | SDK 2.3.0 | TensorFlow | Wikipedia | 66,560 | 16.16 | 12903 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD64 | SDK 2.3.0 | PyTorch | Wikipedia | 65,536 | 16.16 | 12528 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD128 | SDK 2.3.0 | PopART | Wikipedia | 65,536 | 16.16 | 24850 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD128 | SDK 2.3.0 | TensorFlow | Wikipedia | 66,560 | 16.16 | 25097 |
| BERT Large | Ph1 Pre-Training (SL128) | IPU-POD128 | SDK 2.3.0 | PyTorch | Wikipedia | 16.16 | 22971 | |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD16 | SDK 2.3.0 | PopART | Wikipedia | 16,384 | 16.16 | 1055 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD16 | SDK 2.3.0 | TensorFlow | Wikipedia | 16,400 | 16.16 | 1012 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD16 | SDK 2.3.0 | PyTorch | Wikipedia | 16,384 | 16.16 | 984 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD64 | SDK 2.3.0 | PopART | Wikipedia | 16,384 | 16.16 | 3989.9 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD64 | SDK 2.3.0 | TensorFlow | Wikipedia | 16,640 | 16.16 | 3539 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD64 | SDK 2.3.0 | PyTorch | Wikipedia | 16,384 | 16.16 | 3623 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD128 | SDK 2.3.0 | PopART | Wikipedia | 16,384 | 16.16 | 7082 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD128 | SDK 2.3.0 | TensorFlow | Wikipedia | 16,640 | 16.16 | 7366 |
| BERT Large | Ph2 Pre-Training (SL384) | IPU-POD128 | SDK 2.3.0 | PyTorch | Wikipedia | 16.16 | 6579 | |
| BERT Large | Fine-Tuning (SL384 - SQuAD) | IPU-POD16 | SDK 2.3.0 | PopART | SQuAD | 256 | 16.16 | 851 |
| BERT Large | Fine-Tuning (SL384 - SQuAD) | IPU-POD16 | SDK 2.3.0 | PyTorch | SQuAD | 256 | 16.16 | 731 |
| BERT Base | Ph1 Pre-Training (SL128) | IPU-POD16 | SDK 2.3.0 | PopART | Wikipedia | 65,536 | 16.16 | 11763 |
| BERT Base | Ph1 Pre-Training (SL128) | IPU-POD16 | SDK 2.3.0 | TensorFlow | Wikipedia | 65,280 | 16.16 | 11549 |
| BERT Base | Ph1 Pre-Training (SL128) | IPU-POD16 | SDK 2.3.0 | PyTorch | Wikipedia | 65,536 | 16.16 | 11116 |
| BERT Base | Ph2 Pre-Training (SL384) | IPU-POD16 | SDK 2.3.0 | PopART | Wikipedia | 16,384 | 16.16 | 3491 |
| BERT Base | Ph2 Pre-Training (SL384) | IPU-POD16 | SDK 2.3.0 | TensorFlow | Wikipedia | 16,320 | 16.16 | 3250 |
| BERT Base | Ph2 Pre-Training (SL384) | IPU-POD16 | SDK 2.3.0 | PyTorch | Wikipedia | 16,384 | 16.16 | 3265 |
| ResNet-50 v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 1,920 | 16.16 | 7738 | |
| ResNet-50 v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | ImageNet2012 | 16,384 | 16.16 | 7316 | |
| ResNet-50 v1.5 | IPU-POD16 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 1,920 | 16.16 | 29565 | |
| ResNet-50 v1.5 | IPU-POD16 | SDK 2.3.0 | PyTorch | ImageNet2012 | 16,384 | 16.16 | 25382 | |
| ResNet-50 v1.5 | IPU-POD64 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 2,560 | 16.16 | 102320 | |
| ResNet-50 v1.5 | IPU-POD128 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 16.16 | 186553 | ||
| ResNet-50 v1.5 | IPU-POD256 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 16.16 | 355021 | ||
| EfficientNet-B4 | G16-EfficientNet | IPU-POD16 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 800 | 16.32 | 5369 |
| EfficientNet-B4 | G16-EfficientNet | IPU-POD16 | SDK 2.3.0 | PyTorch | ImageNet2012 | 1,024 | 16.32 | 4334 |
| EfficientNet-B4 | G16-EfficientNet | IPU-POD64 | SDK 2.3.0 | TensorFlow | ImageNet2012 | 3,200 | 16.32 | 20951 |
| DeepVoice3 | IPU-M2000 | SDK 2.3.0 | PopART | VCTK Corpus | 128 | 32.32 | 8371 |
训练：结果效率（Time to Result）
| Model | Variant | Platform | SDK Version | Framework | Dataset | Batch Size | Precision | Time To Result (secs) |
|---|---|---|---|---|---|---|---|---|
| MCMC TFP | IPU-M2000 | SDK 2.3.0 | TensorFlow | Proprietary | 32.32 | 52.2 |
推理
此语境中的模型推理是指在输入数据上运行模型以推断输出。生产设置中的推理性能通常通过两个指标来衡量：吞吐量（如前所述）和时延，后者被定义为执行推理所需的时间。
| Model | Variant | Platform | SDK Version | Framework | Dataset | Batch Size | Precision | Throughput (items/sec) | Latency (ms) |
|---|---|---|---|---|---|---|---|---|---|
| BERT-Large | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 4 | 16.16 | 2024 | 1.95 |
| BERT-Large | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 8 | 16.16 | 2840 | 2.78 |
| BERT-Large | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 12 | 16.16 | 3281 | 3.64 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 4 | 16.16 | 4358 | 0.89 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 8 | 16.16 | 6882 | 1.14 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 16 | 16.16 | 9509 | 1.66 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 32 | 16.16 | 12387 | 2.56 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 64 | 16.16 | 15211 | 4.2 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 128 | 16.16 | 17854 | 7.17 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 256 | 16.16 | 19346 | 13.22 |
| BERT-Base | SL128 | IPU-M2000 | SDK 2.3.0 | PopART | SQuAD | 320 | 16.16 | 20653 | 15.48 |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 4 | 16.16 | 6816 | ||
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 16 | 16.16 | 16652 | ||
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 64 | 16.16 | 28695 | ||
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 128 | 16.16 | 34700 | ||
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 256 | 16.16 | 39359 | ||
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 320 | 16.16 | 40341 | ||
| ResNet-50v1.5 | lowest latency config | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 6756 | 0.55 |
| ResNet-50v1.5 | higher throughput config | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 7669 | 0.95 |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 16 | 16.16 | 11951 | 1.22 | |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 32 | 16.16 | 17550 | 1.66 | |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 64 | 16.16 | 29570 | 3.91 | |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 128 | 16.16 | 35388 | 6.44 | |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 256 | 16.16 | 39260 | 11.46 | |
| ResNet-50v1.5 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 360 | 16.16 | 40891 | 15.27 | |
| ResNeXt101 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 4 | 16.16 | 3991 | ||
| ResNeXt101 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 8 | 16.16 | 6140 | ||
| ResNeXt101 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 16 | 16.16 | 9324 | ||
| ResNeXt101 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 32 | 16.16 | 13227 | ||
| ResNeXt101 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 64 | 16.16 | 16808 | ||
| ResNeXt101 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 3295 | 1.17 | |
| EfficientNet-B0 | lowest latency config | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 7697 | 0.47 |
| EfficientNet-B0 | higher throughput config | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 9157 | 0.78 |
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 32 | 16.16 | 37525 | 1.45 | |
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 64 | 16.16 | 50009 | 2.12 | |
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 128 | 16.16 | 56009 | 3.61 | |
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 144 | 16.16 | 55551 | 4.05 | |
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 196 | 16.16 | 62085 | 5.2 | |
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 4 | 16.16 | 9130 | ||
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 32 | 16.16 | 39088 | ||
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 64 | 16.16 | 46755 | ||
| EfficientNet-B0 | IPU-M2000 | SDK 2.3.0 | TensorFlow | Synthetic (host-generated) | 128 | 16.16 | 53994 | ||
| EfficientNet-B4 | lowest latency config | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 3402 | 1.13 |
| EfficientNet-B4 | higher throughput config | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 4 | 16.16 | 3873 | 1.91 |
| EfficientNet-B4 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 16 | 16.16 | 8186 | 3.57 | |
| EfficientNet-B4 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 24 | 16.16 | 9802 | 4.43 | |
| EfficientNet-B4 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 32 | 16.16 | 10752 | 5.35 | |
| EfficientNet-B4 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 40 | 16.16 | 11308 | 6.29 | |
| EfficientNet-B4 | IPU-M2000 | SDK 2.3.0 | PyTorch | Synthetic (host-generated) | 48 | 16.16 | 12091 | 7.08 |
精度术语：X.Y定义如下：X是存储激活和梯度的精度，Y是存储权重的精度。在16.16权重中训练时，我们可能仍将FP32用于其他变量（例如规范或动量），并包括随机舍入。
基准测试是使用我们在 Graphcore GitHub 上的示例生成的。
本页最近更新日期为2021年12月1日
获取最新的GRAPHCORE资讯
在下方注册以获取最新的资讯和更新：