Skip to content

Latest commit

 

History

History
85 lines (80 loc) · 55 KB

README.md

File metadata and controls

85 lines (80 loc) · 55 KB
Raw
[2024-10-28 08:36:34,069 submission_checker1.py:2936 INFO] Results=63, NoResults=0, Power Results=0
[2024-10-28 08:36:34,069 submission_checker1.py:2943 INFO] ---
[2024-10-28 08:36:34,069 submission_checker1.py:2944 INFO] Closed Results=0, Closed Power Results=0

[2024-10-28 08:36:34,069 submission_checker1.py:2949 INFO] Open Results=63, Open Power Results=0

[2024-10-28 08:36:34,069 submission_checker1.py:2954 INFO] Network Results=0, Network Power Results=0

[2024-10-28 08:36:34,069 submission_checker1.py:2959 INFO] ---
[2024-10-28 08:36:34,069 submission_checker1.py:2961 INFO] Systems=60, Power Systems=0
[2024-10-28 08:36:34,069 submission_checker1.py:2962 INFO] Closed Systems=0, Closed Power Systems=0
[2024-10-28 08:36:34,069 submission_checker1.py:2967 INFO] Open Systems=60, Open Power Systems=0
[2024-10-28 08:36:34,069 submission_checker1.py:2972 INFO] Network Systems=0, Network Power Systems=0
[2024-10-28 08:36:34,069 submission_checker1.py:2977 INFO] ---
[2024-10-28 08:36:34,069 submission_checker1.py:2982 INFO] SUMMARY: submission looks OK
INFO:root:       ! call "postprocess" from /home/runner/CM/repos/mlcommons@cm4mlops/script/run-mlperf-inference-submission-checker/customize.py
Organization Availability Division SystemType SystemName Platform Model MlperfModel Scenario Result Accuracy number_of_nodes host_processor_model_name host_processors_per_node host_processor_core_count accelerator_model_name accelerators_per_node Location framework operating_system notes compliance errors version inferred has_power Units weight_data_types
0 scc122 available open edge scc122_cpu0.novalocal scc122_cpu0.novalocal-reference-cpu-deepsparse_v1.8.0-default_config mobilebert-base_quant-none-bert-99 bert-99 Offline 16.1242 F1: 90.81268530031994 1 AMD EPYC-Milan Processor 32 1 nan 0 open/scc122/results/scc122_cpu0.novalocal-reference-cpu-deepsparse_v1.8.0-default_config/mobilebert-base_quant-none-bert-99/offline deepsparse v1.8.0 Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.7. 1 0 v4.1 0 False Samples/s int8
1 scc104-ZJUSCT available open edge scc104_cpu1.novalocal scc104_cpu1.novalocal-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 2.35587 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 32 1 nan 0 open/scc104-ZJUSCT/results/scc104_cpu1.novalocal-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.4. 1 0 v4.1 0 False Samples/s fp32
2 scc104-ZJUSCT available open edge scc104_gpu0.novalocal scc104_gpu0.novalocal-reference-gpu-pytorch-cu122 bert-99 bert-99 Offline 47.9697 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc104-ZJUSCT/results/scc104_gpu0.novalocal-reference-gpu-pytorch-cu122/bert-99/offline pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.4. 1 0 v4.1 0 False Samples/s fp32
3 scc132 available open edge scc132_gpu0.novalocal scc132_gpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 1.96994 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 nan 0 open/scc132/results/scc132_gpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.6. 1 0 v4.1 0 False Samples/s fp32
4 scc107 available open edge mlperf2 mlperf2-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 5.42229 F1: 90.87487229720105 1 AMD EPYC 7B13 1 28 nan 0 open/scc107/results/mlperf2-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Ubuntu 22.04 (linux-6.8.0-1016-gcp-glibc2.35) Automated by MLCommons CM v3.2.7. 1 0 v4.1 0 False Samples/s fp32
5 scc112 available open edge scc112_cpu2.novalocal scc112_cpu2.novalocal-reference-cpu-deepsparse_v1.8.0-default_config mobilebert-base_quant-none-bert-99 bert-99 Offline 99.173 F1: 90.80558974686677 1 AMD EPYC-Milan Processor 32 1 nan 0 open/scc112/results/scc112_cpu2.novalocal-reference-cpu-deepsparse_v1.8.0-default_config/mobilebert-base_quant-none-bert-99/offline deepsparse v1.8.0 Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.5. 1 0 v4.1 0 False Samples/s int8
6 scc131 available open edge scc131_gpu0.novalocal scc131_gpu0.novalocal-reference-gpu-pytorch-cu118 bert-99 bert-99 Offline 47.4872 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc131/results/scc131_gpu0.novalocal-reference-gpu-pytorch-cu118/bert-99/offline pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.3. 1 0 v4.1 0 False Samples/s fp32
7 scc131 available open edge scc131_gpu0.novalocal scc131_gpu0.novalocal-reference-gpu-pytorch-cu118 bert-99 bert-99 SingleStream 20.7943 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc131/results/scc131_gpu0.novalocal-reference-gpu-pytorch-cu118/bert-99/singlestream pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.3. 1 0 v4.1 0 False Latency (ms) fp32
8 scc131 available open edge scc131_gpu0.novalocal scc131_gpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 1.89828 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc131/results/scc131_gpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.3. 1 0 v4.1 0 False Samples/s fp32
9 scc131 available open edge scc131_gpu0.novalocal scc131_gpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 SingleStream 579.913 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc131/results/scc131_gpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config/bert-99/singlestream pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.3. 1 0 v4.1 0 False Latency (ms) fp32
10 MLCommons available open datacenter 87891b324cb3 87891b324cb3-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.1417 CLIP_SCORE: 15.683020070195198 FID_SCORE: 235.19071113978646 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/87891b324cb3-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
11 MLCommons available open datacenter c62ef8beba30 c62ef8beba30-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.18075 CLIP_SCORE: 15.425002038478851 FID_SCORE: 235.8521718479492 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/c62ef8beba30-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v3.2.5. 1 0 v4.1 0 False Samples/s int8
12 MLCommons available open datacenter d714f54675c3 d714f54675c3-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.13923 CLIP_SCORE: 15.476153820753098 FID_SCORE: 238.0083764446639 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/d714f54675c3-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
13 MLCommons available open datacenter ad34f02bda42 ad34f02bda42-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.17889 CLIP_SCORE: 15.444628179073334 FID_SCORE: 235.2164504985768 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/ad34f02bda42-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v3.2.5. 1 0 v4.1 0 False Samples/s int8
14 MLCommons available open datacenter ba096989170c ba096989170c-reference-gpu-pytorch_v2.5.0-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.369623 CLIP_SCORE: 15.180260419845581 FID_SCORE: 235.62720741859 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/ba096989170c-reference-gpu-pytorch_v2.5.0-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.5.0 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v3.2.4. 1 0 v4.1 0 False Samples/s fp32
15 MLCommons available open datacenter 3c7f9df24e39 3c7f9df24e39-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.14095 CLIP_SCORE: 15.649463787674904 FID_SCORE: 233.9612743344963 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/3c7f9df24e39-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
16 MLCommons available open edge gh_action gh_action-reference-gpu-pytorch_v2.4.1-default_config stable-diffusion-xl stable-diffusion-xl Offline 0.345721 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.4.1-default_config/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.4. 1 0 v4.1 0 False Samples/s fp32
17 MLCommons available open edge gh_action gh_action-reference-gpu-pytorch_v2.4.1-default_config gptj-99 gptj-99 Offline 52.9478 nan 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.4.1-default_config/gptj-99/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.4. 1 0 v4.1 0 False Tokens/s fp32
18 MLCommons available open datacenter b5c0891ab663 b5c0891ab663-reference-gpu-pytorch_v2.5.0-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.369148 CLIP_SCORE: 15.180260419845581 FID_SCORE: 235.62720741859 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/b5c0891ab663-reference-gpu-pytorch_v2.5.0-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.5.0 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v3.2.5. 1 0 v4.1 0 False Samples/s fp32
19 MLCommons available open datacenter b1fcd0c956f2 b1fcd0c956f2-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.377055 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/b1fcd0c956f2-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
20 MLCommons available open datacenter 41485dfb4f36 41485dfb4f36-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.375138 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/41485dfb4f36-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
21 MLCommons available open datacenter a2437f3a512c a2437f3a512c-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.375614 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/a2437f3a512c-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
22 MLCommons available open datacenter 05c9134fd82d 05c9134fd82d-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.13939 CLIP_SCORE: 15.569556221365929 FID_SCORE: 232.78986158959376 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/05c9134fd82d-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
23 MLCommons available open datacenter e44bdf36b499 e44bdf36b499-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.37689 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/e44bdf36b499-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
24 MLCommons available open datacenter 897760fb5a09 897760fb5a09-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.18002 CLIP_SCORE: 15.661652073264122 FID_SCORE: 232.0085402303848 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/897760fb5a09-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v3.2.4. 1 0 v4.1 0 False Samples/s int8
25 MLCommons available open datacenter 48ed6105bd85 48ed6105bd85-nvidia-gpu-TensorRT-scc24-main stable-diffusion-xl stable-diffusion-xl Offline 1.13292 CLIP_SCORE: 15.586050063371658 FID_SCORE: 236.8087101317688 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/48ed6105bd85-nvidia-gpu-TensorRT-scc24-main/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.6. 1 0 v4.1 0 False Samples/s int8
26 MLCommons available open datacenter c933bf5bd417 c933bf5bd417-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.13966 CLIP_SCORE: 15.615820929408073 FID_SCORE: 236.01606844647543 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/c933bf5bd417-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
27 MLCommons available open datacenter 6e3eecbcf50d 6e3eecbcf50d-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.376963 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/6e3eecbcf50d-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
28 MLCommons available open datacenter 48ed6105bd85 48ed6105bd85-reference-gpu-pytorch_v2.1.0a0-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 0.373636 CLIP_SCORE: 15.236237794160843 FID_SCORE: 238.78369342212613 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/48ed6105bd85-reference-gpu-pytorch_v2.1.0a0-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.6. 1 0 v4.1 0 False Samples/s fp32
29 MLCommons available open datacenter 97c3ec750d0e 97c3ec750d0e-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.14052 CLIP_SCORE: 15.478164181113243 FID_SCORE: 232.17873111420926 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/97c3ec750d0e-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
30 MLCommons available open datacenter 814f0bc3f38f 814f0bc3f38f-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.376351 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/814f0bc3f38f-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
31 MLCommons available open datacenter f9ac88850adc f9ac88850adc-reference-gpu-pytorch_v2.4.1-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 0.376944 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/f9ac88850adc-reference-gpu-pytorch_v2.4.1-scc24-base/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
32 MLCommons available open datacenter 2ea99d9fdd48 2ea99d9fdd48-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.176157 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/2ea99d9fdd48-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
33 MLCommons available open datacenter 48ed6105bd85 48ed6105bd85-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.13598 CLIP_SCORE: 15.586050063371658 FID_SCORE: 236.8087101317688 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/48ed6105bd85-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.6. 1 0 v4.1 0 False Samples/s int8
34 MLCommons available open datacenter 13fce262fb79 13fce262fb79-reference-gpu-pytorch_v2.4.1-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 0.375843 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/13fce262fb79-reference-gpu-pytorch_v2.4.1-scc24-base/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
35 MLCommons available open datacenter 53f27922dbd5 53f27922dbd5-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.376164 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/53f27922dbd5-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
36 MLCommons available open datacenter 3b07702db56d 3b07702db56d-reference-gpu-pytorch_v2.4.1-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 0.374549 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/3b07702db56d-reference-gpu-pytorch_v2.4.1-scc24-base/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
37 MLCommons available open datacenter 415f0692c960 415f0692c960-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.375643 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/415f0692c960-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
38 MLCommons available open datacenter a51568200dc1 a51568200dc1-reference-gpu-pytorch_v2.4.1-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.375287 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/a51568200dc1-reference-gpu-pytorch_v2.4.1-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
39 MLCommons available open datacenter 2c11da85618f 2c11da85618f-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.14155 CLIP_SCORE: 15.78126573562622 FID_SCORE: 240.29271177824552 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/2c11da85618f-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
40 MLCommons available open datacenter e8dbfdd7ca14 e8dbfdd7ca14-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.13976 CLIP_SCORE: 15.617164582014084 FID_SCORE: 233.28573786792805 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 1 open/MLCommons/results/e8dbfdd7ca14-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s int8
41 MLCommons available open edge gh_action gh_action-reference-gpu-pytorch_v2.4.1-cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.345763 CLIP_SCORE: 15.18544016778469 FID_SCORE: 235.69504308101006 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.4.1-cu124/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v2.3.9. 1 0 v4.1 0 False Samples/s fp32
42 MLCommons available open datacenter 705b75a6e915 705b75a6e915-reference-gpu-pytorch_v2.5.0-scc24-base_cu124 stable-diffusion-xl stable-diffusion-xl Offline 0.369758 CLIP_SCORE: 15.180260419845581 FID_SCORE: 235.62720741859 1 Intel(R) Xeon(R) w7-2495X 1 24 NVIDIA GeForce RTX 4090 2 open/MLCommons/results/705b75a6e915-reference-gpu-pytorch_v2.5.0-scc24-base_cu124/stable-diffusion-xl/offline pytorch v2.5.0 Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) Automated by MLCommons CM v3.2.5. 1 0 v4.1 0 False Samples/s fp32
43 scc135 available open edge scc135_gpu0.novalocal scc135_gpu0.novalocal-reference-gpu-pytorch-cu118 bert-99 bert-99 Offline 48.0956 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc135/results/scc135_gpu0.novalocal-reference-gpu-pytorch-cu118/bert-99/offline pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.7. 1 0 v4.1 0 False Samples/s fp32
44 scc135 available open edge scc135_cpu1.novalocal scc135_cpu1.novalocal-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 3.35681 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 32 1 nan 0 open/scc135/results/scc135_cpu1.novalocal-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Rocky 9.4 (linux-6.1.112-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.7. 1 0 v4.1 0 False Samples/s fp32
45 scc102 available open edge scc102_gpu0.novalocal scc102_gpu0.novalocal-reference-gpu-pytorch-cu118 bert-99 bert-99 Offline 46.2795 F1: 88.74428571428572 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc102/results/scc102_gpu0.novalocal-reference-gpu-pytorch-cu118/bert-99/offline pytorch Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.8. 1 0 v4.1 0 False Samples/s fp32
46 scc123-Shanxi_University available open edge b7828419c93b b7828419c93b-reference-gpu-pytorch-cu124 bert-99 bert-99 Offline 83.4708 F1: 90.87601905331469 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/scc123-Shanxi_University/results/b7828419c93b-reference-gpu-pytorch-cu124/bert-99/offline pytorch Ubuntu 22.04 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.35) Automated by MLCommons CM v3.2.6. 1 0 v4.1 0 False Samples/s fp32
47 scc123-Shanxi_University available open edge scc123_cpu0.novalocal scc123_cpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 3.26888 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 32 1 nan 0 open/scc123-Shanxi_University/results/scc123_cpu0.novalocal-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Rocky 9.4 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.34) Automated by MLCommons CM v3.2.6. 1 0 v4.1 0 False Samples/s fp32
48 UNM-Roadrunners available open datacenter 4751fbf98487 4751fbf98487-reference-cpu-pytorch_v2.4.1-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 0.0144109 CLIP_SCORE: 15.140117883682251 FID_SCORE: 238.40571124804268 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 1 open/UNM-Roadrunners/results/4751fbf98487-reference-cpu-pytorch_v2.4.1-scc24-base/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.35) Automated by MLCommons CM v3.0.1. 1 0 v4.1 0 False Samples/s fp32
49 UNM-Roadrunners available open datacenter d8727d7820f0 d8727d7820f0-nvidia-gpu-TensorRT-scc24-main stable-diffusion-xl stable-diffusion-xl Offline 1.61009 CLIP_SCORE: 15.495350807905197 FID_SCORE: 233.7000707826474 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 4 open/UNM-Roadrunners/results/d8727d7820f0-nvidia-gpu-TensorRT-scc24-main/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.31) Automated by MLCommons CM v2.4.0. 1 0 v4.1 0 False Samples/s int8
50 UNM-Roadrunners available open datacenter 7b06989ff188 7b06989ff188-nvidia-gpu-TensorRT-scc24-main stable-diffusion-xl stable-diffusion-xl Offline 1.61013 CLIP_SCORE: 15.527581945061684 FID_SCORE: 239.8460356754585 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 4 open/UNM-Roadrunners/results/7b06989ff188-nvidia-gpu-TensorRT-scc24-main/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.31) Automated by MLCommons CM v2.4.0. 1 0 v4.1 0 False Samples/s int8
51 UNM-Roadrunners available open datacenter 4751fbf98487 4751fbf98487-reference-gpu-pytorch_v2.4.1-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 0.666646 CLIP_SCORE: 15.170220836997032 FID_SCORE: 235.68987775810245 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 1 open/UNM-Roadrunners/results/4751fbf98487-reference-gpu-pytorch_v2.4.1-scc24-base/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.35) Automated by MLCommons CM v3.0.1. 1 0 v4.1 0 False Samples/s fp32
52 UNM-Roadrunners available open datacenter d8727d7820f0 d8727d7820f0-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.61345 CLIP_SCORE: 15.495350807905197 FID_SCORE: 233.70007081366145 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 4 open/UNM-Roadrunners/results/d8727d7820f0-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.31) Automated by MLCommons CM v2.4.0. 1 0 v4.1 0 False Samples/s int8
53 UNM-Roadrunners available open datacenter 7b06989ff188 7b06989ff188-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.61853 CLIP_SCORE: 15.527581945061684 FID_SCORE: 239.84603575864136 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 4 open/UNM-Roadrunners/results/7b06989ff188-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.31) Automated by MLCommons CM v2.4.0. 1 0 v4.1 0 False Samples/s int8
54 UNM-Roadrunners available open datacenter 4751fbf98487 4751fbf98487-reference-gpu-pytorch_v2.4.1-scc24-main stable-diffusion-xl stable-diffusion-xl Offline 0.665545 CLIP_SCORE: 13.906862080842256 FID_SCORE: 84.21082273715388 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 1 open/UNM-Roadrunners/results/4751fbf98487-reference-gpu-pytorch_v2.4.1-scc24-main/stable-diffusion-xl/offline pytorch v2.4.1 Ubuntu 22.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.35) Automated by MLCommons CM v3.0.1. 1 0 v4.1 0 False Samples/s fp32
55 UNM-Roadrunners available open datacenter roadrunner roadrunner-nvidia-gpu-TensorRT-scc24-base stable-diffusion-xl stable-diffusion-xl Offline 1.60725 CLIP_SCORE: 15.527581945061684 FID_SCORE: 239.84603550136694 1 INTEL(R) XEON(R) PLATINUM 8580 2 60 NVIDIA H100 NVL 4 open/UNM-Roadrunners/results/roadrunner-nvidia-gpu-TensorRT-scc24-base/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-5.14.0-427.37.1.el9_4.x86_64-glibc2.31) Automated by MLCommons CM v2.4.0. 1 0 v4.1 0 False Samples/s int8
56 NTUHPC available open datacenter 8297ae0eca20 8297ae0eca20-nvidia-gpu-TensorRT-scc24-main stable-diffusion-xl stable-diffusion-xl Offline 2.66695 CLIP_SCORE: 15.704176977276802 FID_SCORE: 233.5689613089101 1 AMD EPYC 9654 96-Core Processor 2 96 NVIDIA H100 PCIe 4 open/NTUHPC/results/8297ae0eca20-nvidia-gpu-TensorRT-scc24-main/stable-diffusion-xl/offline TensorRT Ubuntu 20.04 (linux-6.5.0-27-generic-glibc2.31) Automated by MLCommons CM v2.4.0. 1 0 v4.1 0 False Samples/s int8
57 SYSU-LPC-Expects available open edge 210fdd372d21 210fdd372d21-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 3.76647 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 32 1 nan 0 open/SYSU-LPC-Expects/results/210fdd372d21-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Ubuntu 22.04 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.35) Automated by MLCommons CM v3.2.6. 1 0 v4.1 0 False Samples/s fp32
58 SYSU-LPC-Expects available open edge b2b4f72c06b8 b2b4f72c06b8-nvidia-gpu-TensorRT-default_config bert-99 bert-99 Offline 3470.62 F1: 90.43411501470122 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/SYSU-LPC-Expects/results/b2b4f72c06b8-nvidia-gpu-TensorRT-default_config/bert-99/offline TensorRT Ubuntu 20.04 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.31) Automated by MLCommons CM v3.2.4. 1 0 v4.1 0 False Samples/s int8
59 SYSU-LPC-Expects available open edge c1979786bc1c c1979786bc1c-reference-cpu-pytorch_v2.5.0-default_config bert-99 bert-99 Offline 2.30891 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 16 1 nan 0 open/SYSU-LPC-Expects/results/c1979786bc1c-reference-cpu-pytorch_v2.5.0-default_config/bert-99/offline pytorch v2.5.0 Ubuntu 22.04 (linux-6.1.110-1.el9.elrepo.x86_64-glibc2.35) Automated by MLCommons CM v3.2.5. 1 0 v4.1 0 False Samples/s fp32
60 Dreambrook_Team available open edge 0e7e43cc4195 0e7e43cc4195-reference-gpu-pytorch-cu124 bert-99 bert-99 Offline 279.352 F1: 90.8791726326272 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/Dreambrook_Team/results/0e7e43cc4195-reference-gpu-pytorch-cu124/bert-99/offline pytorch Ubuntu 22.04 (linux-6.1.112-1.el9.elrepo.x86_64-glibc2.35) Automated by MLCommons CM v3.2.2. 1 0 v4.1 0 False Samples/s fp32
61 Dreambrook_Team available open edge a65eca5827dc a65eca5827dc-reference-cpu-pytorch_v2.4.1-default_config bert-99 bert-99 Offline 3.19276 F1: 90.87487229720105 1 AMD EPYC-Milan Processor 32 1 nan 0 open/Dreambrook_Team/results/a65eca5827dc-reference-cpu-pytorch_v2.4.1-default_config/bert-99/offline pytorch v2.4.1 Ubuntu 22.04 (linux-6.1.112-1.el9.elrepo.x86_64-glibc2.35) Automated by MLCommons CM v3.2.1. 1 0 v4.1 0 False Samples/s fp32
62 Dreambrook_Team available open edge 666dfb9811e4 666dfb9811e4-reference-gpu-pytorch-cu124 bert-99 bert-99 Offline 85.4467 F1: 90.87601905331469 1 AMD EPYC-Milan Processor 16 1 GRID A100X-20C 1 open/Dreambrook_Team/results/666dfb9811e4-reference-gpu-pytorch-cu124/bert-99/offline pytorch Ubuntu 22.04 (linux-6.1.112-1.el9.elrepo.x86_64-glibc2.35) Automated by MLCommons CM v3.2.2. 1 0 v4.1 0 False Samples/s fp32