Performance Review V0.1
To gauge the practical viability of the current state of our privacy solution, Mobius, we have devised a set of basic functional tests of the performance of the fundamental components of Mobius. As of the time of writing we have been able to demonstrate a proof of concept of a single payment originating from a sender of some address being arbitrated through a ring which, once filled, can be withdrawn by some arbitrary recipient. Our performance review benchmarks this flow of payment at each atomic stage of interaction on the blockchain; a deposit and a withdrawal. There are other off-chain processes that we may need to consider at a more mature stage of the project, but currently we focus on on-chain processes and it's effects and performance on the network itself.
The main core of Mobius is the Mixer where deposits and withdrawals are made. The Mixer contract accepts deposits and creates rings of certain size and payment denomination to store the funds. Once the ring is full, it can be withdrawn from by a recipient who would subsequently perform a withdraw call to the Mixer to retrieve the funds. Each call of Deposit and Withdraw causes the contract to perform computation which will cost both gas and time. We intend to gather data on the performance of Mobius in this context to ascertain the current theoretical bounds of performance that the core functionality of Mobius can currently achieve.
We are testing on a private network that runs proof of authority consensus algorithms with a block time of 5s and block gas limit of 4712388. Test are performed with different ring sizes of 2, 4, 8 and 16. We are currently transacting payments of 2^20 Wei to and from the Mixer as a constant.
During testing we found behaviour that we deemed worth separating:
- Initial Deposits that invoke the creation of a new ring
- 'Normal' Deposits that deposit into an existing ring and do not fill it
- Final Deposits that deposit into an existing and fill it
- Final withdrawals that destroy the ring when left empty after withdrawal
- 'Normal' Withdrawals that leave the ring non-empty after withdrawal
Initial Deposits would consume more gas due to the creation and storage of a new ring.
Final Deposits would consume more gas due to the emission of MixerReady events and the subsequent information carried with it.
Final withdrawals would consume less gas due to the refunding of gas upon the destruction of storage of the ring.
Thereby, each Withdraw and Deposit benchmarking result has been separated into two and three respective further distinctions in order to properly monitor the rate of change of each different action with different ring sizes on our network. We assume that dissimilar actions may change in a non-linear manner dependent on many factors including network configuration, ring size and network load at the time of action. In which case, we aim to track as many possible variations to more accurately discern the behaviour of Mobius under different circumstances.
For benchmarking purposes, a specific, vanilla Mixer contract was deployed to measure the computation costs and time spent performing a simple contract function call supplied with associated variables in order to accurately gauge the statistics of the execution of the functions and any network effects. This was performed on the two functions being tested: deposit and withdraw which we will reference as raw deposit and raw withdrawal. The raw results are calculated from an average of 10 attempts for each function per ring with a total of 40 attempts.
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 10025 ±751 | 26873 ±0 |
Table 1.1: Average gas and time spent for a simple empty deposit function call with data
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9903 ±540 | 24389 ±0 |
Table 1.2: Average gas and time spent for a simple empty withdraw function call with data
The results above should be used alongside the results shown below as a relative comparison between gas and time spent. This should outline how much time and gas is actually being used in the computation of the Mixer contract.
This ring size would not be recommended for use due to the small size of the ring providing little obfuscation of payment but was included to examine the statistics in tandem with other ring sizes.
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9779 ±773 | 213186 ±7532 |
Table 2.1: Average gas and time spent for the initial deposit to ring of size 2
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 10213 ±141 | 147230 ±36461 |
Table 2.2: Average gas and time spent for the final deposit to ring of size 2
Graph 2.3: Average gas and time spent for deposits to ring of size 2
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9799 ±816 | 435267 ±96 |
Table 2.4: Average gas and time spent for normal withdrawal from ring of size 2
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 10018 ±563 | 290759 ±64 |
Table 2.5: Average gas and time spent for the final withdrawal from ring of size 2
Graph 2.6: Average gas and time spent for withdrawals from ring of size 2
This ring size is our current default.
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9949 ±543 | 213177 ±7532 |
Table 3.1: Average gas and time spent for the initial deposit to ring of size 4
| Deposit No. | Average Time Spent (ms) | Average Gas Usage |
|---|---|---|
| 2 | 9800 ±856 | 105028 ±32 |
| 3 | 10125 ±572 | 105819 ±32 |
Table 3.2: Average gas and time spent for normal deposits to ring of size 4
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 10066 ±574 | 160457 ±14629 |
Table 3.3: Average gas and time spent for the final deposit to ring of size 4
Graph 3.4: Average gas and time spent for deposits to ring of size 4
| Withdrawal No. | Average Time Spent (ms) | Average Gas Usage |
|---|---|---|
| 1 | 9734 ±580 | 787561 ±96 |
| 2 | 9951 ±120 | 773312 ±128 |
| 3 | 9888 ±740 | 774018 ±96 |
Table 3.5: Average gas and time spent for normal withdrawals from ring of size 4
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 10070 ±736 | 566486 ±96 |
Table 3.6: Average gas and time spent for the final withdrawal from ring of size 4
Graph 3.7: Average gas and time spent for withdrawals from ring of size 4
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9815 ±599 | 213190 ±7532 |
Table 4.1: Average gas and time spent for the initial deposit to ring of size 8
| Deposit No. | Average Time Spent (ms) | Average Gas Usage |
|---|---|---|
| 2 | 9906 ±558 | 105028 ±32 |
| 3 | 9914 ±419 | 105812 ±32 |
| 4 | 10018 ±587 | 106609 ±32 |
| 5 | 10457 ±560 | 107380 ±32 |
| 6 | 9948 ±69 | 108184 ±0 |
| 7 | 10067 ±1081 | 108961 ±32 |
Table 4.2: Average gas and time spent for normal deposits to ring of size 8
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9830 ±69 | 166500 ±14597 |
Table 4.3: Average gas and time spent for the final deposit to ring of size 8
Graph 4.4: Average gas and time spent for deposits to ring of size 8
| Withdrawal No. | Average Time Spent (ms) | Average Gas Usage |
|---|---|---|
| 1 | 8703 ±542 | 1492244 ±192 |
| 2 | 9991 ±393 | 1478091 ±192 |
| 3 | 9996 ±590 | 1478848 ±64 |
| 4 | 9982 ±560 | 1479547 ±96 |
| 5 | 10002 ±590 | 1480426 ±64 |
| 6 | 9940 ±203 | 1481107 ±128 |
| 7 | 10078 ±739 | 1481889 ±192 |
Table 4.5: Average gas and time spent for normal withdrawals from ring of size 8
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9857 ±525 | 1118201 ±192 |
Table 4.6: Average gas and time spent for the final withdrawal from ring of size 8
Graph 4.7: Average gas and time spent for withdrawals from ring of size 8
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9776 ±823 | 213164 ±7532 |
Table 5.1: Average gas and time spent for the initial deposit to ring of size 16
| Deposit No. | Average Time Spent (ms) | Average Gas Usage |
|---|---|---|
| 2 | 10015 ±585 | 105035 ±64 |
| 3 | 10152 ±593 | 105819 ±32 |
| 4 | 10425 ±698 | 106597 ±32 |
| 5 | 9618 ±742 | 107381 ±32 |
| 6 | 9952 ±626 | 108158 ±64 |
| 7 | 9922 ±612 | 108955 ±456 |
| 8 | 9913 ±518 | 109746 ±32 |
| 9 | 9961 ±230 | 110536 ±0 |
| 10 | 10278 ±614 | 111282 ±32 |
| 11 | 9856 ±789 | 112085 ±64 |
| 12 | 10129 ±898 | 112882 ±32 |
| 13 | 9884 ±462 | 113653 ±32 |
| 14 | 10079 ±568 | 114456 ±392 |
| 15 | 10076 ±512 | 115221 ±32 |
Table 5.2: Average gas and time spent for normal deposits to ring of size 16
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9866 ±1116 | 165480 ±29258 |
Table 5.3: Average gas and time spent for the final deposit to ring of size 16
Graph 5.4: Average gas and time spent for deposits to ring of size 16
| Deposit No. | Average Time Spent (ms) | Average Gas Usage |
|---|---|---|
| 1 | 6627 ±2132 | 2902219 ±192 |
| 2 | 10055 ±231 | 2888034 ±224 |
| 3 | 9906 ±420 | 2888733 ±224 |
| 4 | 10031 ±191 | 2889548 ±641 |
| 5 | 9959 ±80 | 2890440 ±256 |
| 6 | 10234 ±444 | 2891184 ±449 |
| 7 | 9967 ±534 | 2891877 ±449 |
| 8 | 9970 ±31 | 2892698 ±577 |
| 9 | 9972 ±102 | 2893430 ±481 |
| 10 | 9978 ±193 | 2894238 ±609 |
| 11 | 10032 ±514 | 2895002 ±449 |
| 12 | 9978 ±102 | 2895740 ±385 |
| 13 | 9935 ±256 | 2896465 ±449 |
| 14 | 10057 ±309 | 2897293 ±160 |
| 15 | 10125 ±593 | 2898031 ±224 |
Table 5.5: Average gas and time spent for normal withdrawals from ring of size 16
| Average Time Spent (ms) | Average Gas Usage |
|---|---|
| 9987 ±158 | 2221980 ±256 |
Table 5.6: Average gas and time spent for the final withdrawal from ring of size 16
Graph 5.7: Average gas and time spent for withdrawals from ring of size 16
The raw deposit and withdrawal results from tables 1.1 and 1.2 show us that most of the time spent is due to network latency than computation time and almost all time spent performing computation is negligible relative to network effects.
Tables 5.2 and 4.2 show that deposit gas consumption increases with occupation of the ring; as the ring becomes more filled, each subsequent deposit will cost slightly more gas to perform. Initial deposits were significantly more expensive as expected due to the creation of the ring. Final deposits were also more expensive than 'normal' deposits due to emission of data via Mixer Ready event. The difference in gas consumption between initial deposits and final deposits seems to be very similar to the difference in gas consumption between final deposits and 'normal' deposits.
Ring size seems to have no outright effect on the gas consumption of deposits. Deposit costs simply increase as a function of increasing ring occupation as stated above and shown in tables 5.2 and 4.2 (more visibily due to the larger ring size); ring size has an indirect effect on gas consumption as larger rings mean more incrementing of gas consumption.
First withdrawals seem to cost slightly more gas than the other 'normal' withdrawals and also take less time in the larger rings. Final withdrawals were significantly cheaper to perform as expected.
Increasing ring size drastically affects the gas consumption of withdrawals. There seems to be no effect of remaining ring occupation on the gas consumption of subsequent withdrawals. However, it seems that gas consumption of withdrawals increase rougly proportionally with ring size; i.e. withdrawal gas usage for ring size of 16 is about double that of ring size of 8 which is about double that of ring size of 4 etc.
Final withdrawals seem to be around 25% cheaper than other 'normal' withdrawals for all ring sizes due to the emptying of the ring and its deletion from the blockchain storage refunding gas.
For our private network configuration assuming maximum liquidity and flooding the chain with one type of action at a time:
| Ring Size | Maximum Deposits per Block | Maximum Withdrawals per Block |
|---|---|---|
| 2 | 28 | 13 |
| 4 | 32 | 7 |
| 8 | 37 | 4 |
| 16 | 40 | 2 |
Table 6.1: Correlation between ring size and subsequent deposits and withdrawals per block on current network configuration
We can see that as ring size increases, the numerous cheaper 'normal' deposits and less frequent expensive ring creation calls allow more deposits to be fit in a single block. However, the prohibitive rise in the withdrawal costs mean that very few can be subsequently withdrawn per block. With a block time of 5s, it means that once a ring of size 16 is filled, consisting of 32 deposits, it can only be withdrawn in the following block assuming no other activity (aside from a buffer of 8 more deposits before hitting the block gas limit).
Deposits for varying ring sizes seem to behave very similarly and do not introduce significant changes to gas consumption for different ring sizes:
Graph 6.2: Gas consumption per deposit for ring sizes 2, 4, 8 and 16
We can clearly see a trend that can be extrapolated for any ring size here. However the significance of the rising trend of gas consumption relative to the practical amounts of gas usage for successive deposits is negligible. Therefore we can assume that ring size effects on deposits are null and that they remain constant for all ring sizes.
Withdrawals for varying ring sizes indicate, as mentioned above, huge differences in gas consumption outright for each withdrawal depending on the ring size:
Graph 6.3: Gas consumption per withdrawal for ring sizes 2, 4, 8 and 16
The successive difference for withdrawals between each set of rings seems to be proportionally increasing. So we can estimate that for any ring of size n where n is a multiple of 2, where the gas consumption of the first withdrawal for a ring of size 2 is denoted as g, the gas consumption of any withdrawal number less than n will be = gnr/2 where r ≈ 0.87.
Note: r is a multiplier that converges to 0.8 as n increases. For n = 2, this value starts at 1. 0.87 was used as a middlepoint for our spread of n. As such, r = f(n) where f has yet to be formally defined.
For example, for a ring of size n = 8, and g ≈ 435000 (taken from data above), we can estimate the gas consumption for a withdrawal to be around 1513000. Our calculated result is within 3% error of the actual average result range across the withdrawals.
Timing graphs for varying rings have been excluded as we found no real pattern or fluctation in timing of computation of the contract.
Correlating our results with our network configuration:
- Block time of 5s
- Block gas limit of 4712388
and our environment assumptions:
- 100% liquidity; rings are filled immediately and subsequently immediately emptied through withdrawal
- single payment denomination
- single consecutive payments, ring creation and destruction
- single ring size on network
we can suggest that the most optimum balance between through-put of transactions and obfuscation of payments is achieved with a ring size of 4.
Using table 6.1, we observe that a ring size of 8 would restrict a full withdrawal from a ring within a block. This means that if a ring is filled, it would not be able to be withdrawn from until the next block 5s later and no other actions can be performed in that block. As such, moving to the next lower ring size of 4 not only allows a full withdrawal within a block, but a full deposit and withdraw cycle can be performed within a single block. The extra margin also opens up possibility of a greater number of concurrent events in a single block. Higher ring sizes also restrict how many full deposits that can be made per block. Ring sizes of 4, 8 and 16 can fit 8, 4 and 2 full deposits (filled rings) respectively; higher ring sizes would be more prone to splitting of deposits to a ring into separate blocks if we include concurrent actions and where atomicity of transactions is preferred. Higher network activity would thereby favour a lower ring size where a ring size of 4 would be the lowest we could achieve without heavily compromising security.
Using a ring of size 4 with current network configurations we can manage a threoretical maximum through-put of 32 deposits or 7 withdrawals via Mobius per 5 seconds. In practice we have observed a real figure closer to 28 deposits or 5 withdrawals per 5 seconds. This only accounts for single type of actions (deposit or withdrawal) filling a block. If we interchange actions in the same block, we can achieve a full cycle of actions with 4 withdrawals and a minimum of 4 deposits in the same block. In this case however, withdrawals can only be made from deposits that were submitted in a previous block as the blockchain cannot simultaneously handle and validate deposits to be ready for withdraw before the block has been mined.
Depending on proposed network activity, the current performance of Mobius may or may not be sufficient for the expected rate of transactions on the 'real' network. As such, this report provides a basis of findings based on the current state of Mobius and our test network in order to accurately evaluate it's viability and future development path.