From bb086236ae7dc74c6d4593742318aa044ea914a5 Mon Sep 17 00:00:00 2001 From: h5law <53987565+h5law@users.noreply.github.com> Date: Sat, 23 Dec 2023 13:17:21 +0000 Subject: [PATCH 1/5] feat: move benchmarks into docs folder, tidy readme and docs --- README.md | 254 +----------------- docs/benchmarks.md | 254 ++++++++++++++++++ docs/{KVStore.md => kvstore.md} | 20 +- docs/{MerkleSumTrie.md => merkle-sum-trie.md} | 12 +- docs/{SMT.md => smt.md} | 57 ++-- 5 files changed, 316 insertions(+), 281 deletions(-) create mode 100644 docs/benchmarks.md rename docs/{KVStore.md => kvstore.md} (90%) rename docs/{MerkleSumTrie.md => merkle-sum-trie.md} (97%) rename docs/{SMT.md => smt.md} (89%) diff --git a/README.md b/README.md index 3d9807e..a1a6128 100644 --- a/README.md +++ b/README.md @@ -2,34 +2,22 @@ [![Tag](https://img.shields.io/github/v/tag/pokt-network/smt.svg?sort=semver)](https://img.shields.io/github/v/tag/pokt-network/smt.svg?sort=semver) [![GoDoc](https://godoc.org/github.com/pokt-network/smt?status.svg)](https://godoc.org/github.com/pokt-network/smt) +![Go Version](https://img.shields.io/github/go-mod/go-version/pokt-network/smt) [![Go Report Card](https://goreportcard.com/badge/github.com/pokt-network/smt)](https://goreportcard.com/report/github.com/pokt-network/smt) [![Tests](https://github.com/pokt-network/smt/actions/workflows/test.yml/badge.svg)](https://github.com/pokt-network/smt/actions/workflows/test.yml) [![codecov](https://codecov.io/gh/pokt-network/smt/branch/main/graph/badge.svg)](https://codecov.io/gh/pokt-network/smt) -Note: **Requires Go 1.20+** - - [Overview](#overview) - [Documentation](#documentation) - [Tests](#tests) - [Benchmarks](#benchmarks) - - [Definitions](#definitions) - - [Bytes/Operation (B/op)](#bytesoperation-bop) - - [Commit](#commit) - - [Sizing](#sizing) - - [SMT](#smt) - - [Fill](#fill) - - [Operations](#operations) - - [SMST](#smst) - - [Fill](#fill-1) - - [Operations](#operations-1) - - [Proofs](#proofs) - - [SMT](#smt-1) - - [SMST](#smst-1) +**NOTE: Requires Go 1.20.12** + ## Overview This is a Go library that implements a Sparse Merkle Trie for a key-value map. @@ -41,13 +29,12 @@ mechanics. ## Documentation -Documentation for the different aspects of this library can be found in the -[docs](./docs/) directory. +Documentation for the different aspects of this library, the trie, proofs and +all its different components can be found in the [docs](./docs/) directory. ## Tests -To run all the unit tests in the repo (excluding fuzz tests and benchmarks) -simply run the following command: +To run all tests (excluding benchmarks) run the following command: ```sh make test_all @@ -55,237 +42,14 @@ make test_all ## Benchmarks -Benchmarks for the different aspects of this SMT library can be found in -[benchmarks](./benchmarks/). In order to run the entire benchmarking suite use -the following command: +To run the full suite of benchmarks simply run the following command: ```sh make benchmark_all ``` -### Definitions - -Below is a list of terms used in the benchmarks' results that may need -clarification. - -#### Bytes/Operation (B/op) - -- This refers to the number of bytes allocated for each operation. - -#### Commit - -- The `Commit` term refers to the `Commit` method of the trie. This takes all - changes (which are made in memory) to the trie and writes them to the - underlying database. - -#### Sizing - -- The tests use the following sizes: 0.1M, 0.5M, 1M, 5M, 10M. The `M` refers to - millions hence: - - 0.1M = 100,000 (One hundred thousand) - - 0.5M = 500,000 (Five hundred thousand) - - 1M = 1,000,000 (One million) - - 5M = 5,000,000 (Five million) - - 10M = 10,000,000 (Ten million) -- These sizes refer to the number of key-value pairs or key-value-sum triples - inserted into the trie either beforehand or during the benchmark depending on - which benchmark it is. - -_NOTE: Unless otherwise stated the benchmarks in this document were ran on a -2023 14-inch Macbook Pro M2 Max with 32GB of RAM. The tries tested are using the -`sha256.New()` hasher._ - -_TODO: There is an opportunity to do a fuzz test where we commit every `N` -updates, if this ever becomes a bottlneck_ - -### SMT - -In order to run the SMT benchmarks use the following command: - -```sh -make benchmark_smt -``` - -#### Fill - -The "fill" benchmarks cover the time taken to insert `N` key-value pairs into -the SMT, as well as how long it takes to do this and commit these changes to -disk. This gives us an insight into how long it takes to build a trie of a -certain size. - -In order to run the SMT filling benchmarks use the following command: - -```sh -make benchmark_smt_fill -``` - -| Benchmark | # Values | Iterations | Time (s/op) | Bytes (B/op) | Allocations (allocs/op) | -| ------------- | -------- | ---------- | ------------- | --------------- | ----------------------- | -| Fill | 0.1M | 10 | 0.162967196 | 159,479,499 | 2,371,598 | -| Fill & Commit | 0.1M | 10 | 2.877307858 | 972,961,486 | 15,992,605 | -| Fill | 0.5M | 10 | 0.926864771 | 890,408,326 | 13,021,258 | -| Fill & Commit | 0.5M | 10 | 16.043430012 | 5,640,034,396 | 82,075,720 | -| Fill | 1M | 10 | 2.033616088 | 1,860,523,968 | 27,041,639 | -| Fill & Commit | 1M | 10 | 32.617249642 | 12,655,347,004 | 166,879,661 | -| Fill | 5M | 10 | 12.502309738 | 10,229,139,731 | 146,821,675 | -| Fill & Commit | 5M | 10 | 175.421250979 | 78,981,342,709 | 870,235,579 | -| Fill | 10M | 10 | 29.718092496 | 21,255,245,031 | 303,637,210 | -| Fill & Commit | 10M | 10 | 396.142675962 | 173,053,933,624 | 1,775,304,977 | - -#### Operations - -The "operations" benchmarks cover the time taken to perform a single operation -on an SMT of a given size, and also how long doing this operation followed by a -commit would take. This gives us insight into how the SMT operates when filled -to differing degrees. - -In order to run the SMT operation benchmarks use the following command: - -```sh -make benchmark_smt_ops -``` - -| Benchmark | Prefilled Values | Iterations | Time (ns/op) | Bytes (B/op) | Allocations (allocs/op) | -| --------------- | ---------------- | ---------- | ------------ | ------------ | ----------------------- | -| Update | 0.1M | 740,618 | 1,350 | 1,753 | 25 | -| Update & Commit | 0.1M | 21,022 | 54,665 | 13,110 | 281 | -| Update | 0.5M | 605,348 | 1,682 | 1,957 | 26 | -| Update & Commit | 0.5M | 11,697 | 91,028 | 21,501 | 468 | -| Update | 1M | 545,701 | 1,890 | 2,112 | 28 | -| Update & Commit | 1M | 9,540 | 119,347 | 24,983 | 545 | -| Update | 5M | 466,688 | 2,226 | 2,453 | 31 | -| Update & Commit | 5M | 7,906 | 186,026 | 52,621 | 722 | -| Update | 10M | 284,580 | 5,263 | 2,658 | 33 | -| Update & Commit | 10M | 4,484 | 298,376 | 117,923 | 844 | -| Get | 0.1M | 3,923,601 | 303.2 | 48 | 3 | -| Get | 0.5M | 2,209,981 | 577.7 | 48 | 3 | -| Get | 1M | 1,844,431 | 661.6 | 48 | 3 | -| Get | 5M | 1,196,467 | 1,030 | 48 | 3 | -| Get | 10M | 970,195 | 2,667 | 48 | 3 | -| Prove | 0.1M | 829,801 | 1,496 | 2,177 | 17 | -| Prove | 0.5M | 610,402 | 1,835 | 2,747 | 17 | -| Prove | 1M | 605,799 | 1,905 | 2,728 | 17 | -| Prove | 5M | 566,930 | 2,129 | 2,731 | 17 | -| Prove | 10M | 458,472 | 7,113 | 2,735 | 17 | -| Delete | 0.1M | 12,081,112 | 96.18 | 50 | 3 | -| Delete & Commit | 0.1M | 26,490 | 39,568 | 7,835 | 177 | -| Delete | 0.5M | 7,253,522 | 140.3 | 64 | 3 | -| Delete & Commit | 0.5M | 12,766 | 80,518 | 16,696 | 376 | -| Delete | 1M | 1,624,569 | 629.6 | 196 | 4 | -| Delete & Commit | 1M | 9,811 | 135,606 | 20,254 | 456 | -| Delete | 5M | 856,424 | 1,400 | 443 | 6 | -| Delete & Commit | 5M | 8,431 | 151,107 | 74,133 | 626 | -| Delete | 10M | 545,876 | 4,173 | 556 | 6 | -| Delete & Commit | 10M | 3,916 | 271,332 | 108,396 | 772 | - -### SMST - -In order to run the SMST benchmarks use the following command: - -```sh -make benchmark_smst -``` - -#### Fill - -The "fill" benchmarks cover the time taken to insert `N` key-value-sum triples -into the SMST, as well as how long it takes to do this and commit these changes -to disk. This gives us an insight into how long it takes to build a trie of a -certain size. - -In order to run the SMST filling benchmarks use the following command: - -```sh -make benchmark_smst_fill -``` - -| Benchmark | # Values | Iterations | Time (s/op) | Bytes (B/op) | Allocations (allocs/op) | -| ------------- | -------- | ---------- | ------------- | --------------- | ----------------------- | -| Fill | 0.1M | 10 | 0.157951888 | 165,878,234 | 2,471,593 | -| Fill & Commit | 0.1M | 10 | 3.011097462 | 1,058,069,050 | 16,664,811 | -| Fill | 0.5M | 10 | 0.927521862 | 922,408,350 | 13,521,259 | -| Fill & Commit | 0.5M | 10 | 15.338199979 | 6,533,439,773 | 85,880,046 | -| Fill | 1M | 10 | 1.982756162 | 1,924,516,467 | 28,041,610 | -| Fill & Commit | 1M | 10 | 31.197517821 | 14,874,342,889 | 175,474,251 | -| Fill | 5M | 10 | 12.054370871 | 10,549,075,488 | 151,821,423 | -| Fill & Commit | 5M | 10 | 176.912009238 | 89,667,234,678 | 914,653,740 | -| Fill | 10M | 10 | 26.859672362 | 21,894,837,504 | 313,635,611 | -| Fill & Commit | 10M | 10 | 490.805535617 | 197,997,807,905 | 1,865,882,489 | - -#### Operations - -The "operations" benchmarks cover the time taken to perform a single operation -on an SMST of a given size, and also how long doing this operation followed by -a commit would take. This gives us insight into how the SMST operates when -filled to differing degrees. - -In order to run the SMST operation benchmarks use the following command: - -```sh -make benchmark_smst_ops -``` - -| Benchmark | Prefilled Values | Iterations | Time (ns/op) | Bytes (B/op) | Allocations (allocs/op) | -| --------------- | ---------------- | ---------- | ------------ | ------------ | ----------------------- | -| Update | 0.1M | 913,760 | 1,477 | 1,843 | 25 | -| Update & Commit | 0.1M | 20,318 | 49,705 | 13,440 | 256 | -| Update | 0.5M | 687,813 | 1,506 | 1,965 | 27 | -| Update & Commit | 0.5M | 14,526 | 83,295 | 37,604 | 428 | -| Update | 1M | 630,310 | 1,679 | 2,076 | 28 | -| Update & Commit | 1M | 11,678 | 122,568 | 25,760 | 501 | -| Update | 5M | 644,193 | 1,850 | 2,378 | 31 | -| Update & Commit | 5M | 6,214 | 184,533 | 60,755 | 723 | -| Update | 10M | 231,714 | 4,962 | 2,616 | 33 | -| Update & Commit | 10M | 4,284 | 279,893 | 77,377 | 830 | -| Get | 0.1M | 3,924,031 | 281.3 | 40 | 2 | -| Get | 0.5M | 2,080,167 | 559.6 | 40 | 2 | -| Get | 1M | 1,609,478 | 718.6 | 40 | 2 | -| Get | 5M | 1,015,630 | 1,105 | 40 | 2 | -| Get | 10M | 352,980 | 2,949 | 40 | 2 | -| Prove | 0.1M | 717,380 | 1,692 | 2,344 | 18 | -| Prove | 0.5M | 618,265 | 1,972 | 3,040 | 19 | -| Prove | 1M | 567,594 | 2,117 | 3,044 | 19 | -| Prove | 5M | 446,062 | 2,289 | 3,045 | 19 | -| Prove | 10M | 122,347 | 11,215 | 3,046 | 19 | -| Delete | 0.1M | 1,000,000 | 1,022 | 1,110 | 7 | -| Delete & Commit | 0.1M | 1,000,000 | 1,039 | 1,110 | 7 | -| Delete | 0.5M | 1,046,163 | 1,159 | 1,548 | 7 | -| Delete & Commit | 0.5M | 907,071 | 1,143 | 1,548 | 7 | -| Delete | 1M | 852,918 | 1,246 | 1,552 | 8 | -| Delete & Commit | 1M | 807,847 | 1,303 | 1,552 | 8 | -| Delete | 5M | 625,662 | 1,604 | 1,552 | 8 | -| Delete & Commit | 5M | 864,432 | 1,382 | 1,552 | 8 | -| Delete | 10M | 232,544 | 4,618 | 1,552 | 8 | -| Delete & Commit | 10M | 224,767 | 5,048 | 1,552 | 8 | - -### Proofs - -To run the tests to average the proof size for numerous prefilled tries use the -following command: - -```sh -make benchmark_proof_sizes -``` - -#### SMT - -| Prefilled Size | Average Serialised Proof Size (bytes) | Min (bytes) | Max (bytes) | Average Serialised Compacted Proof Size (bytes) | Min (bytes) | Max (bytes) | -| -------------- | ------------------------------------- | ----------- | ----------- | ----------------------------------------------- | ----------- | ----------- | -| 100,000 | 780 | 650 | 1310 | 790 | 692 | 925 | -| 500,000 | 856 | 716 | 1475 | 866 | 758 | 1024 | -| 1,000,000 | 890 | 716 | 1475 | 900 | 758 | 1057 | -| 5,000,000 | 966 | 815 | 1739 | 976 | 858 | 1156 | -| 10,000,000 | 999 | 848 | 1739 | 1010 | 891 | 1189 | - -#### SMST - -| Prefilled Size | Average Serialised Proof Size (bytes) | Min (bytes) | Max (bytes) | Average Serialised Compacted Proof Size (bytes) | Min (bytes) | Max (bytes) | -| -------------- | ------------------------------------- | ----------- | ----------- | ----------------------------------------------- | ----------- | ----------- | -| 100,000 | 935 | 780 | 1590 | 937 | 822 | 1101 | -| 500,000 | 1030 | 862 | 1795 | 1032 | 904 | 1224 | -| 1,000,000 | 1071 | 868 | 1795 | 1073 | 910 | 1265 | -| 5,000,000 | 1166 | 975 | 2123 | 1169 | 1018 | 1388 | -| 10,000,000 | 1207 | 1026 | 2123 | 1210 | 1059 | 1429 | +To view pre-ran results of the entire benchmarking suite see +[benchmarks](./docs/benchmarks.md) [jmt whitepaper]: https://developers.diem.com/papers/jellyfish-merkle-tree/2021-01-14.pdf [libra whitepaper]: https://diem-developers-components.netlify.app/papers/the-diem-blockchain/2020-05-26.pdf diff --git a/docs/benchmarks.md b/docs/benchmarks.md new file mode 100644 index 0000000..30065d1 --- /dev/null +++ b/docs/benchmarks.md @@ -0,0 +1,254 @@ +# Benchmarks + + + +- [Overview](#overview) +- [Definitions](#definitions) + * [Bytes/Operation (B/op)](#bytesoperation-bop) + * [Commit](#commit) + * [Sizing](#sizing) +- [SMT](#smt) + * [Fill](#fill) + * [Operations](#operations) +- [SMST](#smst) + * [Fill](#fill-1) + * [Operations](#operations-1) +- [Proofs](#proofs) + * [SMT](#smt-1) + * [SMST](#smst-1) + + + +## Overview + +Benchmarks for the different aspects of this SMT library can be found in +[benchmarks](../benchmarks/). In order to run the entire benchmarking suite use +the following command: + +```sh +make benchmark_all +``` + +## Definitions + +Below is a list of terms used in the benchmarks' results that may need +clarification. + +### Bytes/Operation (B/op) + +- This refers to the number of bytes allocated for each operation. + +### Commit + +- The `Commit` term refers to the `Commit` method of the trie. This takes all + changes (which are made in memory) to the trie and writes them to the + underlying database. + +### Sizing + +- The tests use the following sizes: 0.1M, 0.5M, 1M, 5M, 10M. The `M` refers to + millions hence: + - 0.1M = 100,000 (One hundred thousand) + - 0.5M = 500,000 (Five hundred thousand) + - 1M = 1,000,000 (One million) + - 5M = 5,000,000 (Five million) + - 10M = 10,000,000 (Ten million) +- These sizes refer to the number of key-value pairs or key-value-sum triples + inserted into the trie either beforehand or during the benchmark depending on + which benchmark it is. + +_NOTE: Unless otherwise stated the benchmarks in this document were ran on a +2023 14-inch Macbook Pro M2 Max with 32GB of RAM. The tries tested are using the +`sha256.New()` hasher._ + +_TODO: There is an opportunity to do a fuzz test where we commit every `N` +updates, if this ever becomes a bottlneck_ + +## SMT + +In order to run the SMT benchmarks use the following command: + +```sh +make benchmark_smt +``` + +### Fill + +The "fill" benchmarks cover the time taken to insert `N` key-value pairs into +the SMT, as well as how long it takes to do this and commit these changes to +disk. This gives us an insight into how long it takes to build a trie of a +certain size. + +In order to run the SMT filling benchmarks use the following command: + +```sh +make benchmark_smt_fill +``` + +| Benchmark | # Values | Iterations | Time (s/op) | Bytes (B/op) | Allocations (allocs/op) | +| ------------- | -------- | ---------- | ------------- | --------------- | ----------------------- | +| Fill | 0.1M | 10 | 0.162967196 | 159,479,499 | 2,371,598 | +| Fill & Commit | 0.1M | 10 | 2.877307858 | 972,961,486 | 15,992,605 | +| Fill | 0.5M | 10 | 0.926864771 | 890,408,326 | 13,021,258 | +| Fill & Commit | 0.5M | 10 | 16.043430012 | 5,640,034,396 | 82,075,720 | +| Fill | 1M | 10 | 2.033616088 | 1,860,523,968 | 27,041,639 | +| Fill & Commit | 1M | 10 | 32.617249642 | 12,655,347,004 | 166,879,661 | +| Fill | 5M | 10 | 12.502309738 | 10,229,139,731 | 146,821,675 | +| Fill & Commit | 5M | 10 | 175.421250979 | 78,981,342,709 | 870,235,579 | +| Fill | 10M | 10 | 29.718092496 | 21,255,245,031 | 303,637,210 | +| Fill & Commit | 10M | 10 | 396.142675962 | 173,053,933,624 | 1,775,304,977 | + +### Operations + +The "operations" benchmarks cover the time taken to perform a single operation +on an SMT of a given size, and also how long doing this operation followed by a +commit would take. This gives us insight into how the SMT operates when filled +to differing degrees. + +In order to run the SMT operation benchmarks use the following command: + +```sh +make benchmark_smt_ops +``` + +| Benchmark | Prefilled Values | Iterations | Time (ns/op) | Bytes (B/op) | Allocations (allocs/op) | +| --------------- | ---------------- | ---------- | ------------ | ------------ | ----------------------- | +| Update | 0.1M | 740,618 | 1,350 | 1,753 | 25 | +| Update & Commit | 0.1M | 21,022 | 54,665 | 13,110 | 281 | +| Update | 0.5M | 605,348 | 1,682 | 1,957 | 26 | +| Update & Commit | 0.5M | 11,697 | 91,028 | 21,501 | 468 | +| Update | 1M | 545,701 | 1,890 | 2,112 | 28 | +| Update & Commit | 1M | 9,540 | 119,347 | 24,983 | 545 | +| Update | 5M | 466,688 | 2,226 | 2,453 | 31 | +| Update & Commit | 5M | 7,906 | 186,026 | 52,621 | 722 | +| Update | 10M | 284,580 | 5,263 | 2,658 | 33 | +| Update & Commit | 10M | 4,484 | 298,376 | 117,923 | 844 | +| Get | 0.1M | 3,923,601 | 303.2 | 48 | 3 | +| Get | 0.5M | 2,209,981 | 577.7 | 48 | 3 | +| Get | 1M | 1,844,431 | 661.6 | 48 | 3 | +| Get | 5M | 1,196,467 | 1,030 | 48 | 3 | +| Get | 10M | 970,195 | 2,667 | 48 | 3 | +| Prove | 0.1M | 829,801 | 1,496 | 2,177 | 17 | +| Prove | 0.5M | 610,402 | 1,835 | 2,747 | 17 | +| Prove | 1M | 605,799 | 1,905 | 2,728 | 17 | +| Prove | 5M | 566,930 | 2,129 | 2,731 | 17 | +| Prove | 10M | 458,472 | 7,113 | 2,735 | 17 | +| Delete | 0.1M | 12,081,112 | 96.18 | 50 | 3 | +| Delete & Commit | 0.1M | 26,490 | 39,568 | 7,835 | 177 | +| Delete | 0.5M | 7,253,522 | 140.3 | 64 | 3 | +| Delete & Commit | 0.5M | 12,766 | 80,518 | 16,696 | 376 | +| Delete | 1M | 1,624,569 | 629.6 | 196 | 4 | +| Delete & Commit | 1M | 9,811 | 135,606 | 20,254 | 456 | +| Delete | 5M | 856,424 | 1,400 | 443 | 6 | +| Delete & Commit | 5M | 8,431 | 151,107 | 74,133 | 626 | +| Delete | 10M | 545,876 | 4,173 | 556 | 6 | +| Delete & Commit | 10M | 3,916 | 271,332 | 108,396 | 772 | + +## SMST + +In order to run the SMST benchmarks use the following command: + +```sh +make benchmark_smst +``` + +### Fill + +The "fill" benchmarks cover the time taken to insert `N` key-value-sum triples +into the SMST, as well as how long it takes to do this and commit these changes +to disk. This gives us an insight into how long it takes to build a trie of a +certain size. + +In order to run the SMST filling benchmarks use the following command: + +```sh +make benchmark_smst_fill +``` + +| Benchmark | # Values | Iterations | Time (s/op) | Bytes (B/op) | Allocations (allocs/op) | +| ------------- | -------- | ---------- | ------------- | --------------- | ----------------------- | +| Fill | 0.1M | 10 | 0.157951888 | 165,878,234 | 2,471,593 | +| Fill & Commit | 0.1M | 10 | 3.011097462 | 1,058,069,050 | 16,664,811 | +| Fill | 0.5M | 10 | 0.927521862 | 922,408,350 | 13,521,259 | +| Fill & Commit | 0.5M | 10 | 15.338199979 | 6,533,439,773 | 85,880,046 | +| Fill | 1M | 10 | 1.982756162 | 1,924,516,467 | 28,041,610 | +| Fill & Commit | 1M | 10 | 31.197517821 | 14,874,342,889 | 175,474,251 | +| Fill | 5M | 10 | 12.054370871 | 10,549,075,488 | 151,821,423 | +| Fill & Commit | 5M | 10 | 176.912009238 | 89,667,234,678 | 914,653,740 | +| Fill | 10M | 10 | 26.859672362 | 21,894,837,504 | 313,635,611 | +| Fill & Commit | 10M | 10 | 490.805535617 | 197,997,807,905 | 1,865,882,489 | + +### Operations + +The "operations" benchmarks cover the time taken to perform a single operation +on an SMST of a given size, and also how long doing this operation followed by +a commit would take. This gives us insight into how the SMST operates when +filled to differing degrees. + +In order to run the SMST operation benchmarks use the following command: + +```sh +make benchmark_smst_ops +``` + +| Benchmark | Prefilled Values | Iterations | Time (ns/op) | Bytes (B/op) | Allocations (allocs/op) | +| --------------- | ---------------- | ---------- | ------------ | ------------ | ----------------------- | +| Update | 0.1M | 913,760 | 1,477 | 1,843 | 25 | +| Update & Commit | 0.1M | 20,318 | 49,705 | 13,440 | 256 | +| Update | 0.5M | 687,813 | 1,506 | 1,965 | 27 | +| Update & Commit | 0.5M | 14,526 | 83,295 | 37,604 | 428 | +| Update | 1M | 630,310 | 1,679 | 2,076 | 28 | +| Update & Commit | 1M | 11,678 | 122,568 | 25,760 | 501 | +| Update | 5M | 644,193 | 1,850 | 2,378 | 31 | +| Update & Commit | 5M | 6,214 | 184,533 | 60,755 | 723 | +| Update | 10M | 231,714 | 4,962 | 2,616 | 33 | +| Update & Commit | 10M | 4,284 | 279,893 | 77,377 | 830 | +| Get | 0.1M | 3,924,031 | 281.3 | 40 | 2 | +| Get | 0.5M | 2,080,167 | 559.6 | 40 | 2 | +| Get | 1M | 1,609,478 | 718.6 | 40 | 2 | +| Get | 5M | 1,015,630 | 1,105 | 40 | 2 | +| Get | 10M | 352,980 | 2,949 | 40 | 2 | +| Prove | 0.1M | 717,380 | 1,692 | 2,344 | 18 | +| Prove | 0.5M | 618,265 | 1,972 | 3,040 | 19 | +| Prove | 1M | 567,594 | 2,117 | 3,044 | 19 | +| Prove | 5M | 446,062 | 2,289 | 3,045 | 19 | +| Prove | 10M | 122,347 | 11,215 | 3,046 | 19 | +| Delete | 0.1M | 1,000,000 | 1,022 | 1,110 | 7 | +| Delete & Commit | 0.1M | 1,000,000 | 1,039 | 1,110 | 7 | +| Delete | 0.5M | 1,046,163 | 1,159 | 1,548 | 7 | +| Delete & Commit | 0.5M | 907,071 | 1,143 | 1,548 | 7 | +| Delete | 1M | 852,918 | 1,246 | 1,552 | 8 | +| Delete & Commit | 1M | 807,847 | 1,303 | 1,552 | 8 | +| Delete | 5M | 625,662 | 1,604 | 1,552 | 8 | +| Delete & Commit | 5M | 864,432 | 1,382 | 1,552 | 8 | +| Delete | 10M | 232,544 | 4,618 | 1,552 | 8 | +| Delete & Commit | 10M | 224,767 | 5,048 | 1,552 | 8 | + +## Proofs + +To run the tests to average the proof size for numerous prefilled tries use the +following command: + +```sh +make benchmark_proof_sizes +``` + +### SMT + +| Prefilled Size | Average Serialised Proof Size (bytes) | Min (bytes) | Max (bytes) | Average Serialised Compacted Proof Size (bytes) | Min (bytes) | Max (bytes) | +| -------------- | ------------------------------------- | ----------- | ----------- | ----------------------------------------------- | ----------- | ----------- | +| 100,000 | 780 | 650 | 1310 | 790 | 692 | 925 | +| 500,000 | 856 | 716 | 1475 | 866 | 758 | 1024 | +| 1,000,000 | 890 | 716 | 1475 | 900 | 758 | 1057 | +| 5,000,000 | 966 | 815 | 1739 | 976 | 858 | 1156 | +| 10,000,000 | 999 | 848 | 1739 | 1010 | 891 | 1189 | + +### SMST + +| Prefilled Size | Average Serialised Proof Size (bytes) | Min (bytes) | Max (bytes) | Average Serialised Compacted Proof Size (bytes) | Min (bytes) | Max (bytes) | +| -------------- | ------------------------------------- | ----------- | ----------- | ----------------------------------------------- | ----------- | ----------- | +| 100,000 | 935 | 780 | 1590 | 937 | 822 | 1101 | +| 500,000 | 1030 | 862 | 1795 | 1032 | 904 | 1224 | +| 1,000,000 | 1071 | 868 | 1795 | 1073 | 910 | 1265 | +| 5,000,000 | 1166 | 975 | 2123 | 1169 | 1018 | 1388 | +| 10,000,000 | 1207 | 1026 | 2123 | 1210 | 1059 | 1429 | diff --git a/docs/KVStore.md b/docs/kvstore.md similarity index 90% rename from docs/KVStore.md rename to docs/kvstore.md index 3fcda9f..78c7c5a 100644 --- a/docs/KVStore.md +++ b/docs/kvstore.md @@ -4,16 +4,16 @@ - [Overview](#overview) - [Implementation](#implementation) - - [In-Memory and Persistent](#in-memory-and-persistent) - - [Store methods](#store-methods) - - [Lifecycle Methods](#lifecycle-methods) - - [Data Methods](#data-methods) - - [Backups](#backups) - - [Restorations](#restorations) - - [Accessor Methods](#accessor-methods) - - [Prefixed and Sorted Get All](#prefixed-and-sorted-get-all) - - [Clear All Key-Value Pairs](#clear-all-key-value-pairs) - - [Len](#len) + * [In-Memory and Persistent](#in-memory-and-persistent) + * [Store methods](#store-methods) + * [Lifecycle Methods](#lifecycle-methods) + * [Data Methods](#data-methods) + + [Backups](#backups) + + [Restorations](#restorations) + * [Accessor Methods](#accessor-methods) + + [Prefixed and Sorted Get All](#prefixed-and-sorted-get-all) + + [Clear All Key-Value Pairs](#clear-all-key-value-pairs) + + [Len](#len) diff --git a/docs/MerkleSumTrie.md b/docs/merkle-sum-trie.md similarity index 97% rename from docs/MerkleSumTrie.md rename to docs/merkle-sum-trie.md index a72017c..260164e 100644 --- a/docs/MerkleSumTrie.md +++ b/docs/merkle-sum-trie.md @@ -4,11 +4,11 @@ - [Overview](#overview) - [Implementation](#implementation) - - [Sum Encoding](#sum-encoding) - - [Digests](#digests) - - [Visualisations](#visualisations) - - [General Trie Structure](#general-trie-structure) - - [Binary Sum Digests](#binary-sum-digests) + * [Sum Encoding](#sum-encoding) + * [Digests](#digests) + * [Visualisations](#visualisations) + + [General Trie Structure](#general-trie-structure) + + [Binary Sum Digests](#binary-sum-digests) - [Sum](#sum) - [Nil Values](#nil-values) - [Example](#example) @@ -47,7 +47,7 @@ In practice the SMST is a wrapper around the SMT with a new field added to the `TrieSpec`: `sumTrie bool` this determines whether the SMT should follow its regular encoding of that of the sum trie. -See: the [SMT documentation](./SMT.md) for the details on how the SMT works. +See: the [SMT documentation](./smt.md) for the details on how the SMT works. The majority of the code relating to the SMST can be found in: diff --git a/docs/SMT.md b/docs/smt.md similarity index 89% rename from docs/SMT.md rename to docs/smt.md index 06ad9e2..603a710 100644 --- a/docs/SMT.md +++ b/docs/smt.md @@ -20,6 +20,7 @@ - [Proofs](#proofs) * [Verification](#verification) * [Closest Proof](#closest-proof) + + [Closest Proof Use Cases](#closest-proof-use-cases) * [Compression](#compression) * [Serialisation](#serialisation) - [Database](#database) @@ -384,21 +385,40 @@ of the trie. Since the `ClosestProof` method takes a hash as input, it is possible to place a leaf in the trie according to the hash's path, if it is known. Depending on -the use case of this function this may expose a vulnerability. _It is not -intendend to be used as a general purpose proof mechanism_. Given two parties: -the prover and the verifier, the verifier should supply the prover with a hash -after the trie has been "closed" and is no longer being updated. The prover -will then generate a `ClosestProof` for a leaf using the corresponding method. -The verifier can subsequently verify that proof for its validity. If however, -the prover were to know the hash prior to "closing" the trie, they could place -a leaf where the method would always guarantee it existence. This form of attack -can only happen due to the method's deterministic behaviour and the prover -knowing the hash before they have "closed" the trie. The intended use of this -method is that the verifier gives the hash only after the prover has closed their -trie and submitted the closed trie's root hash. This enables the verifier to -verify the integrity of the proof (if the trie was changed the root hash would -be different) and also guarantees the pseudo-random proof of inclusion was not -a maliciously placed leaf. +the use case of this function this may expose a vulnerability. **It is not +intendend to be used as a general purpose proof mechanism**, but instead as a +**Commit and Reveal** mechanism, as detailed below. + +#### Closest Proof Use Cases + +The `CloestProof` function is intended for use as a `commit & reveal` mechanism. +Where there are two actors involved, the **prover** and **verifier**. + +_NOTE: Throughout this document, `commitment` of the the trie's root hash is also +referred to as closing the trie, such that no more updates are made to it once +committed._ + +Take the following attack vector (**without** a commit prior to a reveal) into +consideration: + +1. The **prover** identifies the hash the **verifier** intends to check against +1. The **prover** then places a leaf in the tree with a common prefix to the + identified hash +1. Due to the deterministic nature of the `ClosestProof` method this leaf will + **always** be returned given the identified hash. +1. The **verifier** then verifies the revealed `ClosestProof`, in turn validating + a maliciously placed leaf. + +Take the following normal flow (**with** a commit prior to reveal) as + +1. The **prover** commits to the state of their trie and publishes their root + hash, _closing_ their trie. +1. The **verifier** provides a hash to be used in the `commit & reveal` process +1. The **prover** then utilises this hash and the `ClosestProof` method on their + _closed_ trie, producing a `ClosestProof`, thus revealing a deterministic, + yet pseudo-random leaf. +1. The **verifier** then verifies the proof, in turn verifying the commitment + made by the **prover** and the state of the **prover**'s trie. ### Compression @@ -438,7 +458,7 @@ node, **not** the leaf node's value, even when `WithValueHasher(nil)` is used. The node value can be parsed from this value, as the trie `Get` function does by removing the prefix and path bytes from the returned value. -See [KVStore.md](./KVStore.md) for the details of the implementation. +See [kvstore.md](./kvstore.md) for the details of the implementation. ### Data Loss @@ -450,7 +470,7 @@ the `Commit()` function is called and changes are persisted. ## Sparse Merkle Sum Trie This library also implements a Sparse Merkle Sum Trie (SMST), the documentation -for which can be found [here](./MerkleSumTrie.md). +for which can be found [here](./merkle-sum-trie.md). ## Example @@ -472,9 +492,6 @@ func main() { // Ensure the database connection closes defer nodeStore.Stop() - // Initialise the trie - trie := smt.NewSparseMerkleTrie(nodeStore, sha256.New()) - // Update the key "foo" with the value "bar" _ = trie.Update([]byte("foo"), []byte("bar")) From 89dee091259f8fb188a4cd749aba952226a007f5 Mon Sep 17 00:00:00 2001 From: h5law <53987565+h5law@users.noreply.github.com> Date: Sat, 23 Dec 2023 13:55:46 +0000 Subject: [PATCH 2/5] feat: update gitignore --- .gitignore | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index 9d2d03c..aa638db 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,8 @@ -# macos files +# Ignore Mac OS X system files .DS_Store -# goland -.idea/ \ No newline at end of file +# Ignore asdf local tool-versions file +.tool-versions + +# Ignore Goland and JetBrains IDE files +.idea/ From dedc8f3b574ef15311d981f3eca3187978693c52 Mon Sep 17 00:00:00 2001 From: h5law <53987565+h5law@users.noreply.github.com> Date: Sat, 23 Dec 2023 13:56:15 +0000 Subject: [PATCH 3/5] feat: cleanup and update workflow file --- .github/workflows/test.yml | 52 +++++++++++++------------------------- 1 file changed, 17 insertions(+), 35 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 161589b..e981f6a 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -1,4 +1,4 @@ -name: Test & Build +name: Go Tests on: pull_request: @@ -12,7 +12,7 @@ concurrency: cancel-in-progress: true env: - TARGET_GOLANG_VERSION: "1.20.11" + TARGET_GOLANG_VERSION: "1.20.12" jobs: tests: @@ -22,7 +22,7 @@ jobs: steps: - uses: actions/checkout@v3 with: - fetch-depth: "0" # Per https://github.com/ignite/cli/issues/1674#issuecomment-1144619147 + fetch-depth: "0" - name: Setup go uses: actions/setup-go@v4 @@ -39,18 +39,27 @@ jobs: - name: Create coverage report and run tests run: | + # Run each of the tests (excluding benchmarks) outputting the JSON + # to the test_results.json file for usage in later steps. set -euo pipefail - GODEBUG=netdns=cgo go test -p 1 -json ./ -mod=readonly -timeout 8m -race -coverprofile=coverage.txt -covermode=atomic 2>&1 | tee test_results.json + go test -v -json -p 1 ./ -mod=readonly -race -coverprofile=coverage.txt -covermode=atomic 2>&1 | tee -a test_results.json - name: Sanitize test results - # We're utilizing `tee` above which can capture non-json stdout output so we need to remove non-json lines before additional parsing and submitting it to the external github action. + # We're utilizing `tee` above which can capture non-json stdout output + # so we need to remove non-json lines before additional parsing and + # submitting it to the external github action. run: cat test_results.json | jq -c -R 'fromjson? | select(type == "object")' > tmp.json && mv tmp.json test_results.json - name: Output test failures - # Makes it easier to find failed tests so no need to scroll through the whole log. if: ${{ failure() }} run: | - jq --argjson fail_tests "$(jq -c -r 'select(.Action == "fail") | select(.Test) | .Test' test_results.json | jq -R -s -c -r 'split("\n") | map(select(length>0))')" 'select(.Test as $t | ($fail_tests | arrays)[] | select($t == .)) | select(.Output) | .Output' test_results.json | jq -r | sed ':a;N;$!ba;s/\n\n/\n/g' > test_failures.json + jq --argjson fail_tests "$( \ + jq -c -r 'select(.Action == "fail") | select(.Test) | .Test' test_results.json \ + | jq -R -s -c 'split("\n") | map(select(length > 0))' \ + )" \ + 'select(.Test as $t | ($fail_tests | arrays)[] | select($t == .)) | select(.Output) | .Output' test_results.json \ + | jq -r \ + | sed ':a;N;$!ba;s/\n\n/\n/g' > test_failures.json cat test_failures.json exit 1 @@ -62,7 +71,6 @@ jobs: test_*.json - name: Annotate tests on GitHub - # Only annotate if the test failed on target version to avoid duplicated annotations on GitHub. uses: guyarb/golang-test-annotations@v0.5.1 with: test-results: test_results.json @@ -71,30 +79,4 @@ jobs: uses: codecov/codecov-action@v3 with: files: ./coverage.txt - - build: - runs-on: ubuntu-latest - needs: tests - strategy: - fail-fast: false - matrix: - goarch: ["arm64", "amd64"] - timeout-minutes: 5 - name: Build for ${{ matrix.goarch }} - steps: - - uses: actions/setup-go@v4 - with: - go-version: ${{ env.TARGET_GOLANG_VERSION }} - - - uses: actions/checkout@v3 - - - uses: technote-space/get-diff-action@v4 - with: - PATTERNS: | - **/**.go - go.mod - go.sum - - - name: Go build - run: GOOS=linux GOARCH=${{ matrix.goarch }} go build - if: env.GIT_DIFF + verbose: true From 2d215eb1c9855271f8aa2f27cdac8953a3bef0e6 Mon Sep 17 00:00:00 2001 From: h5law <53987565+h5law@users.noreply.github.com> Date: Sat, 23 Dec 2023 13:56:37 +0000 Subject: [PATCH 4/5] feat: make codecov reports informational only --- .github/codecov.yml | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 .github/codecov.yml diff --git a/.github/codecov.yml b/.github/codecov.yml new file mode 100644 index 0000000..2c43681 --- /dev/null +++ b/.github/codecov.yml @@ -0,0 +1,9 @@ +# Only post coverage comments and not pass/fail reports in the CI checks +coverage: + status: + project: + default: + informational: true + patch: + default: + informational: true From 179f61687f2e98d83931eb526b363f409d45d2b6 Mon Sep 17 00:00:00 2001 From: h5law <53987565+h5law@users.noreply.github.com> Date: Wed, 3 Jan 2024 14:06:26 +0000 Subject: [PATCH 5/5] chore: address comments --- README.md | 2 +- docs/smt.md | 42 ++++++++++++++++++++++++------------------ 2 files changed, 25 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index a1a6128..2298a88 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ -**NOTE: Requires Go 1.20.12** +**NOTE: Requires Go 1.20.12+** ## Overview diff --git a/docs/smt.md b/docs/smt.md index 603a710..2479f2d 100644 --- a/docs/smt.md +++ b/docs/smt.md @@ -398,27 +398,33 @@ _NOTE: Throughout this document, `commitment` of the the trie's root hash is als referred to as closing the trie, such that no more updates are made to it once committed._ -Take the following attack vector (**without** a commit prior to a reveal) into -consideration: +Consider the following attack vector (**without** a commit prior to a reveal) +into consideration: -1. The **prover** identifies the hash the **verifier** intends to check against -1. The **prover** then places a leaf in the tree with a common prefix to the - identified hash -1. Due to the deterministic nature of the `ClosestProof` method this leaf will +1. The **verifier** picks the hash (i.e. a single branch) they intend to check +1. The **prover** inserts a leaf (i.e. a value) whose key (determined via the + hasher) has a longer common prefix than any other leaf in the trie. +1. Due to the deterministic nature of the `ClosestProof`, method this leaf will **always** be returned given the identified hash. -1. The **verifier** then verifies the revealed `ClosestProof`, in turn validating - a maliciously placed leaf. - -Take the following normal flow (**with** a commit prior to reveal) as - -1. The **prover** commits to the state of their trie and publishes their root - hash, _closing_ their trie. -1. The **verifier** provides a hash to be used in the `commit & reveal` process -1. The **prover** then utilises this hash and the `ClosestProof` method on their +1. The **verifier** then verifies the revealed `ClosestProof`, which returns a + branch the **prover** inserted after knowing which leaf was going to be + checked. + +Consider the following normal flow (**with** a commit prior to reveal) as + +1. The **prover** commits to the state of their trie by publishes their root + hash, thereby _closing_ their trie and not being able to make further + changes. +1. The **verifier** selects a hash to be used in the `commit & reveal` process + that the **prover** must provide a closest proof for. +1. The **prover** utilises this hash and computes the `ClosestProof` on their _closed_ trie, producing a `ClosestProof`, thus revealing a deterministic, - yet pseudo-random leaf. -1. The **verifier** then verifies the proof, in turn verifying the commitment - made by the **prover** and the state of the **prover**'s trie. + pseudo-random leaf that existed in the tree prior to commitment, yet +1. The **verifier** verifies the proof, in turn, verifying the commitment + made by the **prover** to the state of the trie in the first step. +1. The **prover** had no opportunity to insert a new leaf into the trie + after learning which hash the **verifier** was going to require a + `ClosestProof` for. ### Compression