Skip to content

Istanbul BFT Benchmarking

Jump to bottom
Miya Chen edited this page Dec 20, 2017 · 3 revisions

Setup

We run load test with golang test tool and deploy containers to kubernetes cluster by helm. Please make sure there is a prepared kubernetes cluster and go, kubectl and helm are already installed in your enviroment.

To get better test result, each validator deployed to kubernetes node needs to run with enough resources. If the number of nodes in kubernetes cluster is less than the number of deployed validators, that means at least two of validators will be deplyed on one node and these two validators will share the resource of the node they deployed to. You can check the number of nodes in kubernetes cluster by kubectl get nodes.

Run load test

The load test deploys 4 validators and uses IBFT consensus by default. Run load test as the following steps:

NOTE: it might take few minutes to deploy and initial validators.

$ git clone [email protected]:getamis/istanbul-tools.git
$ cd ./istanbul-tools
$ make load-testing

Check the deployed status by the following commands:

  1. Check kubernetes pods status:
$ kubectl get pods
NAME                            READY     STATUS    RESTARTS   AGE
validator-0-0                   1/1       Running   0          1m
validator-1-0                   1/1       Running   0          2m
validator-2-0                   1/1       Running   0          1m
validator-3-0                   1/1       Running   0          1m
  1. Check helm charts status:
$ helm ls
NAME            REVISION        UPDATED                         STATUS          CHART                   NAMESPACE
genesis-block   1               Tue Dec 19 16:22:24 2017        DEPLOYED        genesis-block-0.1.0     default
static-nodes    1               Tue Dec 19 16:22:37 2017        DEPLOYED        static-nodes-0.1.0      default
validator-0     1               Tue Dec 19 16:22:50 2017        DEPLOYED        validator-0.2.3         default
validator-1     1               Tue Dec 19 16:22:45 2017        DEPLOYED        validator-0.2.3         default
validator-2     1               Tue Dec 19 16:22:49 2017        DEPLOYED        validator-0.2.3         default
validator-3     1               Tue Dec 19 16:22:50 2017        DEPLOYED        validator-0.2.3         default
  1. Interrupt load test:

Try Ctrl+c to interrupt the load test and clear remaining services.

Use helm to force delete all resources.

$ helm del --purge genesis-block static-nodes validator-0 validator-1 validator-2 validator-3

Test metrics

The test result will be printed out. You can find the test metrics at the end of testing.

  • Total number of transactions: The following metric presents the total number of sent transactions is 81920.
counter tx/sent
  count:           81920
  • TPS of this load test: The following metric presents the peak tps is 835.
histogram tx/tps/response/all
  count:              25
  min:                 0
  max:               835
  mean:        4.218000e+02
  stddev:      3.074906e+02
  median:      5.140000e+02
  75%:         7.030000e+02
  90%:         7.992000e+02
  95%:         8.311000e+02
  99%:         8.350000e+02

Load test arguments

There are many test arguments in load test, try to input different values of arguments to test the performance.

  • Number of validators

    • How many validators join in IBFT consensus. (default: 4)

  • Gas limit of block

    • Gas limit sets the initial gas limit for the blocks to mine. (default: 31500000)

  • Size of transaction pool

    • The buffer size of transactions not mined yet. (default: 20480)

  • Rate of sending transaction

    • The period in milliseconds to send transactions for each account. (default: 300)
    • NOTE: the validator might run consensus slowly if sending transactions very fast. The possible reason is there might be a lock mechanism of StateDB when verifying transaction and committing new consensus block at the same time.

Reference:

Clone this wiki locally