Skip to content

Commit

Permalink
Merge pull request #484 from ethersphere/postage-stamp-revision
Browse files Browse the repository at this point in the history 
Postage stamp revision
  • Loading branch information
@NoahMaizels
NoahMaizels authored Sep 27, 2023
2 parents 6f3440d + bb6aeb6 commit 73df1d3
Showing 1 changed file with 55 additions and 32 deletions.
87 changes: 55 additions & 32 deletions docs/learn/technology/contracts/postage-stamp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,57 +6,51 @@ id: postage-stamp
import DepthCalc from '@site/src/components/DepthCalc';
import BatchCostCalc from '@site/src/components/BatchCostCalc';

The [postage stamp contract](https://github.com/ethersphere/storage-incentives/blob/master/src/PostageStamp.sol) is a smart contract which is a key part of Swarm's [storage incentives](/docs/learn/technology/incentives) which make up the foundation of Swarm's self-sustaining economic system. It allows users to purchase batches of postage stamps for xBZZ tokens. These stamps can then be used to attach to [chunks](/docs/learn/technology/DISC) that are uploaded to the Swarm network.
The [postage stamp contract](https://github.com/ethersphere/storage-incentives/blob/master/src/PostageStamp.sol) is a smart contract which is a key part of Swarm's [storage incentives](/docs/learn/technology/incentives) which make up the foundation of Swarm's self-sustaining economic system.

When a node uploads data to Swarm, it attaches postage stamps to each chunk of data. Postage stamps are issued in batches rather than one by one. The value assigned to a stamp indicates how much it is worth to persist the associated data on Swarm, which nodes use to prioritize which chunks to remove from their reserve first.
When a node uploads data to Swarm, it attaches postage stamps to each [chunk](/docs/learn/technology/DISC) of data. Postage stamps are issued in batches rather than one by one. The value assigned to a stamp indicates how much it is worth to persist the associated data on Swarm, which nodes use to prioritize which chunks to remove from their reserve first.

The value of a postage stamp decreases over time as if storage rent was regularly deducted from the batch balance. Once the value of a stamp is no longer sufficient, the associated chunk is evicted from the reserve.

## Batch Buckets

Postage stamps are issued in batches with a certain number of storage slots divided amongst $$2^{bucketDepth}$$ equally sized address space buckets. Each bucket is responsible for storing chunks that fall within a certain range of the address space. When uploaded, files are split into 4kb chunks, each chunk is assigned a unique address, and each chunk is then assigned to the bucket in which its address falls. While the value of `bucket depth` is not defined in The Book of Swarm, in its current implementation in the Bee client, `bucket depth` has been set to 16, so there are a total of 65,536 buckets.

## Bucket Depth
### Bucket Size

Postage stamps are issued in batches with a certain number of storage slots divided amongst $$2^{bucketDepth}$$ equally sized address space buckets. Each bucket is responsible for storing chunks that fall within a certain range of address space. When uploaded, files are split into 4kb chunks, each chunk is assigned a unique address, and each chunk is then assigned to the bucket in which its address falls. While the value of `bucket depth` is not defined in The Book of Swarm, in its current implementation in the Bee client, `bucket depth` has been set to 16, so there are a total of 65,536 buckets.
Each bucket has a certain number of slots which can be "filled" by chunks (In other words, for each bucket, a certain number of chunks can be stamped). Once all the slots of a bucket are filled, the entire postage batch will be fully utilised and can no longer be used to upload additional data. Given the constant `bucket depth` of 16, for a `batch depth` of 20, the number of chunks per bucket is calculated like so:

## Batch Depth and Batch Amount

Each batch of stamps has two key parameters, `batch depth` and `amount`, which are recorded on Gnosis Chain at issuance. Note that these "depths" do not refer to the depth terms used to describe topology which are outlined [here in the glossary](/docs/learn/glossary#depth-types).

### Batch Depth
$$
2^{(20 - 16)} = 16 \text{ chunks/bucket}
$$

`Batch depth` is a measurement of how much data can be stored by a batch. The number of chunks which can be stored (stamped) by a batch is equal to $$2^{depth}$$.

`Batch depth` determines how many chunks are allowed in each bucket. The number of chunks allowed in each bucket is calculated like so:

$$
2^{(batchDepth - bucketDepth)}
$$

For a batch with a `batch depth` of 20, a maximum of $$2^{20} = 1,048,576$$ chunks can be stamped.

Given the constant `bucket depth` of 16, for a `batch depth` of 20, the number of chunks per bucket is calculated like so:
## Batch Depth and Batch Amount

$$
2^{(20 - 16)} = 16 \text{ chunks/bucket}
$$
Each batch of stamps has two key parameters, `batch depth` and `amount`, which are recorded on Gnosis Chain at issuance. Note that these "depths" do not refer to the depth terms used to describe topology which are outlined [here in the glossary](/docs/learn/glossary#depth-types).

Since we know that one chunk can store 4 kb of data, we can easily calculate the maximum amount of data (due to [the way batches are utilised](/docs/learn/technology/contracts/postage-stamp#batch-utilisation), postage batches may be fully utilised before storing the maximum amount of data) which can be stored by a batch from the `batch depth`:
### Batch Depth

$$
\text{Maximum data in a batch} = 2^{batchdepth} \times \text{4 kb}
$$
`Batch depth` determines how much data can be stored by a batch. The number of chunks which can be stored (stamped) by a batch is equal to $$2^{depth}$$.

For a batch with a `batch depth` of 23, a maximum of $$2^{23} = 33,554,432$$ chunks can be stamped.

For a batch of depth 16, for example, the maximum amount of data in kilobytes can be calculated like so:
Since we know that one chunk can store 4 kb of data, we can calculate the theoretical maximum amount of data which can be stored by a batch from the `batch depth`.

$$
2^{16} \times \text{4 kb} = 65536 \text{ chunks} \times \text{4 kb} = \text{262144 kb}
\text{Theoretical maximum batch volume} = 2^{batchDepth} \times \text{4 kb}
$$

#### Depth to kb calculator:

<DepthCalc></DepthCalc>
However, due to the way postage stamp batches are utilised, batches will become fully utilised before stamping the theoretical maximum number of chunks. Therefore when deciding which batch depth to use, it is important to consider the effective amount of data that can be stored by a batch, and not the theoretical maximum. The effective rate of utilisation increases along with the batch depth. See [section on stamp batch utilisation below](/docs/learn/technology/contracts/postage-stamp#batch-utilisation) for more information.

### Batch Amount
### Batch Amount (& Batch Cost)

The `amount` parameter is the quantity of xBZZ in PLUR $$(1 \times 10^{16}PLUR = 1 \text{ xBZZ})$$ that is assigned per chunk in the batch. The total number of xBZZ that will be paid for the batch is calculated from this figure and the `batch depth` like so:

Expand Down Expand Up @@ -115,28 +109,57 @@ The default batch type when unspecified is immutable. This can be modified throu

### Implications for Swarm Users

Due to the nature of batch utilisation described above, batches are often fully utilised before reaching their theoretical maximum storage amount. However as the batch depth increases, the chance of a postage batch becoming fully utilised early decreases. At at batch depth 23, it is statistically highly unlikely that the batch will be fully utilised/start replacing old chunks before reaching 50% of its theoretical maximum. Therefore, when choosing a batch depth, it is recommended to use a batch depth of 23 or greater, and consider 50% of its theoretical maximum storage amount as the de facto maximum storage amount.
Due to the nature of batch utilisation described above, batches are often fully utilised before reaching their theoretical maximum storage amount. However as the batch depth increases, the chance of a postage batch becoming fully utilised early decreases. At batch depth 23, there is a 0.1% chance that a batch will be fully utilised/start replacing old chunks before reaching 50% of its theoretical maximum.

Let's look at an example to make it clearer. Using the method of calculating the theoretical maximum storage amount [outlined above](/docs/learn/technology/contracts/postage-stamp#batch-depth), we can see that for a batch depth of 23, the maximum amount which can be stored is 33,554,432 kb:
Let's look at an example to make it clearer. Using the method of calculating the theoretical maximum storage amount [outlined above](/docs/learn/technology/contracts/postage-stamp#batch-depth), we can see that for a batch depth of 23, the theoretical maximum amount which can be stored is 33,554,432 kb:

$$
2^{23} \times \text{4kb} = \text{33,554,432 kb}
$$

Therefore we should use 50% of that value as our de facto storage limit:
Therefore we should use 50% the effective rate of usage for the stamp batch:

$$
\text{33,554,432 kb} \times{0.5} = \text{16,777,216 kb}
\text{33,554,432 kb} \times{0.5} = \text{16,777,216 kb} = \text{16.77 gb}
$$

We find that at depth 23, we have a statistical guarantee of being able to store up to 16,777,216 kb or 16.77 gb worth of data.
:::info
The details of how the effective rates of utilisation are calculated will be published soon.
:::


### Effective Utilisation Table

:::info
More information about the expected utilisation levels for different batch depths will be added in the near future.
This table is based on preliminary calculations and may be subject to change.
:::


The provided table shows the effective volume for each batch depth from 20 to 41. The "utilisation rate" is the rate of utilisation a stamp batch can reach with a 0.1% failure rate (that is, there is a 1/1000 chance the batch will become fully utilised before reaching that utilisation rate). The "effective volume" figure shows the actual amount of data which can be stored at the effective rate. The effective volume figure is the one which should be used as the de-facto maximum amount of data that a batch can store before becoming either fully utilised (for immutable batches), or start overwriting older chunks (mutable batches).

| Batch Depth | Utilisation Rate | Effective Volume | Theoretical Max Volume |
|-------------|------------------|------------------|------------------------|
| 20 | 0.00% | 4.29 GB | 0.00 B |
| 21 | 0.00% | 8.59 GB | 0.00 B |
| 22 | 28.67% | 17.18 GB | 4.93 GB |
| 23 | 49.56% | 34.36 GB | 17.03 GB |
| 24 | 64.33% | 68.72 GB | 44.21 GB |
| 25 | 74.78% | 137.44 GB | 102.78 GB |
| 26 | 82.17% | 274.88 GB | 225.86 GB |
| 27 | 87.39% | 549.76 GB | 480.43 GB |
| 28 | 91.08% | 1.10 TB | 1.00 TB |
| 29 | 93.69% | 2.20 TB | 2.06 TB |
| 30 | 95.54% | 4.40 TB | 4.20 TB |
| 31 | 96.85% | 8.80 TB | 8.52 TB |
| 32 | 97.77% | 17.59 TB | 17.20 TB |
| 33 | 98.42% | 35.18 TB | 34.63 TB |
| 34 | 98.89% | 70.37 TB | 69.58 TB |
| 35 | 99.21% | 140.74 TB | 139.63 TB |
| 36 | 99.44% | 281.47 TB | 279.91 TB |
| 37 | 99.61% | 562.95 TB | 560.73 TB |
| 38 | 99.72% | 1.13 PB | 1.12 PB |
| 39 | 99.80% | 2.25 PB | 2.25 PB |
| 40 | 99.86% | 4.50 PB | 4.50 PB |
| 41 | 99.90% | 9.01 PB | 9.00 PB |



0 comments on commit 73df1d3

Please sign in to comment.