Skip to content

Commit

Permalink
docs(benchmark): update Poseidon2 benchmarks with new flags
Browse files Browse the repository at this point in the history
  • Loading branch information
@ashjeong
ashjeong committed Oct 23, 2024
1 parent a0e82f1 commit 19e7ca3
Show file tree
Hide file tree
Showing 7 changed files with 71 additions and 69 deletions.
140 changes: 71 additions & 69 deletions benchmark/poseidon2/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

```
Run on 13th Gen Intel(R) Core(TM) i9-13900K (32 X 5500 MHz CPU s)
Compiler: clang-15
CPU Caches:
L1 Data 48 KiB (x16)
L1 Instruction 32 KiB (x16)
Expand All @@ -15,47 +16,48 @@ CPU Caches:
L2 Unified 4096 KiB (x12)
```

Note: Run with `build --@rules_rust//:extra_rustc_flags="-Ctarget-cpu=native"` in your .bazelrc.user
Note that Poseidon2 runs 10000x per test due to some time results being too small when running a single iteration.

## BN254

```shell
bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p bn254_fr --vendor horizen --vendor plonky3 --check_results
GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p bn254_fr --vendor horizen --vendor plonky3 --check_results
```

### On Intel i9-13900K

| Trial Number | Tachyon | Horizen | Plonky3 |
| :----------- | -------- | ------------ | -------- |
| 0 | 0.064726 | **0.050047** | 0.082464 |
| 1 | 0.061723 | **0.049997** | 0.082598 |
| 2 | 0.060917 | **0.050063** | 0.08257 |
| 3 | 0.06086 | **0.049952** | 0.082493 |
| 4 | 0.060655 | **0.050173** | 0.082409 |
| 5 | 0.060768 | **0.050683** | 0.08268 |
| 6 | 0.060843 | **0.050278** | 0.082675 |
| 7 | 0.060696 | **0.050062** | 0.082579 |
| 8 | 0.060688 | **0.05004** | 0.082592 |
| 9 | 0.060677 | **0.050128** | 0.083144 |
| avg | 0.061255 | **0.050142** | 0.08262 |
| 0 | 0.069228 | **0.050903** | 0.085679 |
| 1 | 0.062046 | **0.050892** | 0.085772 |
| 2 | 0.06053 | **0.050848** | 0.08553 |
| 3 | 0.060648 | **0.050825** | 0.085643 |
| 4 | 0.060553 | **0.051126** | 0.08583 |
| 5 | 0.060592 | **0.05362** | 0.085475 |
| 6 | 0.060576 | **0.050936** | 0.085731 |
| 7 | 0.06051 | **0.05081** | 0.085613 |
| 8 | 0.060561 | **0.050889** | 0.086382 |
| 9 | 0.060558 | **0.050896** | 0.086557 |
| avg | 0.06158 | **0.051174** | 0.085821 |

![image](/benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png)

### On Mac M3 Pro

| Trial Number | Tachyon | Horizen | Plonky3 |
| :----------- | -------- | ------------ | -------- |
| 0 | 0.072979 | **0.055031** | 0.081624 |
| 1 | 0.072478 | **0.054731** | 0.081525 |
| 2 | 0.071973 | **0.055768** | 0.081633 |
| 3 | 0.071969 | **0.054835** | 0.081638 |
| 4 | 0.072009 | **0.054884** | 0.081545 |
| 5 | 0.071933 | **0.055** | 0.081572 |
| 6 | 0.07201 | **0.054946** | 0.081521 |
| 7 | 0.072033 | **0.054883** | 0.081539 |
| 8 | 0.071967 | **0.054989** | 0.081626 |
| 9 | 0.071934 | **0.054942** | 0.081556 |
| avg | 0.072128 | **0.055** | 0.081577 |
| 0 | 0.068967 | **0.058728** | 0.086994 |
| 1 | 0.068786 | **0.05839** | 0.086825 |
| 2 | 0.068658 | **0.058245** | 0.086779 |
| 3 | 0.068673 | **0.058189** | 0.086693 |
| 4 | 0.068675 | **0.058303** | 0.08674 |
| 5 | 0.068693 | **0.058109** | 0.08681 |
| 6 | 0.068621 | **0.058405** | 0.086816 |
| 7 | 0.068747 | **0.058247** | 0.086871 |
| 8 | 0.068637 | **0.058383** | 0.086842 |
| 9 | 0.068665 | **0.058162** | 0.086846 |
| avg | 0.068712 | **0.058316** | 0.086821 |

![image](/benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png)

Expand All @@ -66,83 +68,83 @@ Note: Horizen and Plonky3 compute values with a different internal matrix, requi
### Horizen

```shell
bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor horizen --check_results
GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor horizen --check_results
```

#### On Intel i9-13900K

| Trial Number | Tachyon | Horizen |
| :----------- | ------------ | -------- |
| 0 | **0.011549** | 0.034751 |
| 1 | **0.011439** | 0.034627 |
| 2 | **0.011475** | 0.034581 |
| 3 | **0.011543** | 0.035442 |
| 4 | **0.011455** | 0.03632 |
| 5 | **0.011372** | 0.034545 |
| 6 | **0.011381** | 0.034538 |
| 7 | **0.011142** | 0.03459 |
| 8 | **0.010845** | 0.034522 |
| 9 | **0.010819** | 0.034589 |
| avg | **0.011302** | 0.03485 |
| 0 | **0.011424** | 0.034238 |
| 1 | **0.011975** | 0.034214 |
| 2 | **0.011505** | 0.034245 |
| 3 | **0.011304** | 0.03418 |
| 4 | **0.011313** | 0.034231 |
| 5 | **0.011354** | 0.034234 |
| 6 | **0.010743** | 0.034487 |
| 7 | **0.01071** | 0.034259 |
| 8 | **0.010706** | 0.034229 |
| 9 | **0.010708** | 0.034246 |
| avg | **0.011174** | 0.034256 |

![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png)

#### On Mac M3 Pro

| Trial Number | Tachyon | Horizen |
| :----------- | ------------ | -------- |
| 0 | **0.010979** | 0.013892 |
| 1 | **0.010574** | 0.013858 |
| 2 | **0.010544** | 0.014098 |
| 3 | **0.010642** | 0.013843 |
| 4 | **0.010517** | 0.013842 |
| 5 | **0.010599** | 0.013938 |
| 6 | **0.010519** | 0.013913 |
| 7 | **0.010474** | 0.013889 |
| 8 | **0.010572** | 0.013892 |
| 9 | **0.010533** | 0.013979 |
| avg | **0.010595** | 0.013914 |
| 0 | **0.010679** | 0.014511 |
| 1 | **0.010448** | 0.014653 |
| 2 | **0.010286** | 0.014961 |
| 3 | **0.01024** | 0.014769 |
| 4 | **0.010233** | 0.014717 |
| 5 | **0.010267** | 0.014761 |
| 6 | **0.010226** | 0.01514 |
| 7 | **0.010303** | 0.01475 |
| 8 | **0.010253** | 0.014693 |
| 9 | **0.010326** | 0.014533 |
| avg | **0.010326** | 0.014748 |

![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png)

### Plonky3

```shell
bazel run --config opt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor plonky3 --check_results
GOMP_SPINCOUNT=0 bazel run --config maxopt --//:has_matplotlib //benchmark/poseidon2:poseidon2_benchmark -- -p baby_bear --vendor plonky3 --check_results
```

#### On Intel i9-13900K

| Trial Number | Tachyon | Plonky3 |
| :----------- | -------- | ------------ |
| 0 | 0.009848 | **0.006655** |
| 1 | 0.00992 | **0.006582** |
| 2 | 0.009955 | **0.006616** |
| 3 | 0.009811 | **0.006572** |
| 4 | 0.009851 | **0.006537** |
| 5 | 0.009776 | **0.006645** |
| 6 | 0.009822 | **0.006548** |
| 7 | 0.009738 | **0.006586** |
| 8 | 0.009757 | **0.006594** |
| 9 | 0.009717 | **0.006619** |
| avg | 0.009819 | **0.006595** |
| 0 | 0.00999 | **0.005391** |
| 1 | 0.009882 | **0.005298** |
| 2 | 0.009848 | **0.00513** |
| 3 | 0.009772 | **0.005157** |
| 4 | 0.00977 | **0.005072** |
| 5 | 0.009774 | **0.005032** |
| 6 | 0.009783 | **0.005062** |
| 7 | 0.009878 | **0.005077** |
| 8 | 0.009778 | **0.005014** |
| 9 | 0.009762 | **0.005016** |
| avg | 0.009823 | **0.005124** |

![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png)

#### On Mac M3 Pro

| Trial Number | Tachyon | Plonky3 |
| :----------- | -------- | ------------ |
| 0 | 0.009201 | **0.00732** |
| 1 | 0.008927 | **0.007346** |
| 2 | 0.008922 | **0.007309** |
| 3 | 0.008735 | **0.007324** |
| 4 | 0.008765 | **0.0076** |
| 5 | 0.008715 | **0.007335** |
| 6 | 0.008704 | **0.007439** |
| 7 | 0.008664 | **0.007369** |
| 8 | 0.008679 | **0.007347** |
| 9 | 0.008657 | **0.007353** |
| avg | 0.008796 | **0.007374** |
| 0 | 0.009116 | **0.007311** |
| 1 | 0.008967 | **0.007352** |
| 2 | 0.008805 | **0.007312** |
| 3 | 0.008748 | **0.007315** |
| 4 | 0.008742 | **0.007339** |
| 5 | 0.008741 | **0.007309** |
| 6 | 0.008774 | **0.00732** |
| 7 | 0.00873 | **0.007696** |
| 8 | 0.008791 | **0.007342** |
| 9 | 0.008741 | **0.007353** |
| avg | 0.008815 | **0.007364** |

![image](/benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png)\*\*\*\*
Binary file modified benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_mac_m3.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified benchmark/poseidon2/poseidon2_benchmark_baby_bear_horizen_ubuntu_i9.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_mac_m3.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified benchmark/poseidon2/poseidon2_benchmark_baby_bear_plonky3_ubuntu_i9.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified benchmark/poseidon2/poseidon2_benchmark_bn254_mac_m3.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified benchmark/poseidon2/poseidon2_benchmark_bn254_ubuntu_i9.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 19e7ca3

Please sign in to comment.