Skip to content

Commit

Permalink
Added additional descriptions of the implications of batch utilisatio…
Browse files Browse the repository at this point in the history
…n with examples
  • Loading branch information
NoahMaizels committed Sep 20, 2023
1 parent 627c155 commit 2fb8368
Showing 1 changed file with 44 additions and 1 deletion.
45 changes: 44 additions & 1 deletion docs/learn/technology/contracts/postage-stamp.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ $$
2^{(20 - 16)} = 16 \text{ chunks/bucket}
$$

Since we know that one chunk can store 4 kb of data, we can easily calculate the maximum amount of data (due to [the way batches are utilised](/docs/learn/technology/contracts/postage-stamp#batch-utilisation), postage batches may be fully utilised before storing the maximum amount of data) which can be stored by a batch from the `batch depth`:

$$
\text{Maximum data in a batch} = 2^{batchdepth} \times \text{4 kb}
$$

For a batch of depth 16, for example, the maximum amount of data in kilobytes can be calculated like so:

$$
2^{16} \times \text{4 kb} = 65536 \text{ chunks} \times \text{4 kb} = \text{262144 kb}
$$

#### Depth to kb calculator:

Expand Down Expand Up @@ -87,12 +98,44 @@ As more chunks get uploaded and stamped, the bucket slots will begin to fill. As

### Mutable Batches

Mutable batches use the same hash map structure as immutable batches, however its utilisation works very differently. In contrast with immutable batches, mutable batches are never considered fully utilised. Rather, at the point where an immutable batch would be considered fully utilised, a mutable batch can continue to stamp chunks. However, if any chunk's address lands in a bucket whose slots are already filled, that bucket's counter gets reset, and the new chunk will replace the oldest chunk in that bucket.
Mutable batches use the same hash map structure as immutable batches, however its utilisation works very differently. In contrast with immutable batches, mutable batches are never considered fully utilised. Rather, at the point where an immutable batch would be considered fully utilised, a mutable batch can continue to stamp chunks. However, if any chunk's address lands in a bucket whose slots are already filled, rather than the batch becoming fully utilised, that bucket's counter gets reset, and the new chunk will replace the oldest chunk in that bucket.

![](/img/batches_04.png)

Therefore rather than speaking of the number of slots as determining the utilisation of a batch as with immutable batches, we can think of the slots as defining a limit to the amount of data which can be uploaded before old data starts to get overwritten.

### Which Type of Batch to Use

Immutable batches are suitable for long term storage of data or for data which otherwise does not need to be changed and should never be overwritten, such as records archival, legal documents, family photos, etc.

Mutable batches are great for data which needs to be frequently updated and does not require a guarantee of immutability. For example, a blog, personal or company websites, ephemeral messaging app, etc.

The default batch type when unspecified is immutable. This can be modified through the Bee api by setting the `immutable` header with the [`\stamps POST` endpoint](https://docs.ethswarm.org/api/#tag/Transaction/paths/~1transactions~1%7BtxHash%7D/post) to `false`.


### Implications for Swarm Users

Due to the nature of batch utilisation described above, batches are often fully utilised before reaching their theoretical maximum storage amount. However as the batch depth increases, the chance of a postage batch becoming fully utilised early decreases. At at batch depth 23, it is statistically highly unlikely that the batch will be fully utilised/start replacing old chunks before reaching 50% of its theoretical maximum. Therefore, when choosing a batch depth, it is recommended to use a batch depth of 23 or greater, and consider 50% of its theoretical maximum storage amount as the de facto maximum storage amount.

Let's look at an example to make it clearer. Using the method of calculating the theoretical maximum storage amount [outlined above](/docs/learn/technology/contracts/postage-stamp#batch-depth), we can see that for a batch depth of 23, the maximum amount which can be stored is 33,554,432 kb:

$$
2^{23} \times \text{4kb} = \text{33,554,432 kb}
$$

Therefore we should use 50% of that value as our de facto storage limit:

$$
\text{33,554,432 kb} \times{0.5} = \text{16,777,216 kb}
$$

We find that at depth 23, we have a statistical guarantee of being able to store up to 16,777,216 kb or 16.77 gb worth of data.


:::info
More information about the expected utilisation levels for different batch depths will be added in the near future.
:::




Expand Down

0 comments on commit 2fb8368

Please sign in to comment.