Merge pull request #479 from ethersphere/batches

Batches

docs/develop/access-the-swarm/keep-your-data-alive.md → docs/develop/access-the-swarm/buy-a-stamp-batch.md

@@ -1,9 +1,9 @@
 ---
-title: Keep Your Data Alive
-id: keep-your-data-alive
+title: Buy a Batch of Stamps
+id: buy-a-stamp-batch
 ---
 
-Swarm comprises the sum total of all storage space provided by all of our nodes, called the DISC (Distributed Immutable Store of Chunks). The _right to write_ data into this distributed store is determined by the postage stamps that have been attached.
+Swarm comprises the sum total of all storage space provided by all of our nodes, called the DISC (Distributed Immutable Store of Chunks). The _right to write_ data into this distributed store is determined by the [postage stamps](/docs/learn/technology/contracts/postage-stamp) that have been attached.
 
 ### Fund your node's wallet.
 

docs/develop/access-the-swarm/upload-and-download.md

@@ -28,7 +28,7 @@ In order to upload your data to swarm, you must agree to burn (spend)
 some of your xBZZ to signify to storer and fowarder nodes that this
 content is valued. Before you proceed to the next step, you must buy
 stamps! See this guide on how to [purchase an appropriate batch of
-stamps](/docs/develop/access-the-swarm/keep-your-data-alive).
+stamps](/docs/develop/access-the-swarm/buy-a-stamp-batch).
 
 ### Upload
 

docs/learn/technology/contracts/postage-stamp.md

@@ -40,6 +40,17 @@ $$
 2^{(20 - 16)} = 16 \text{ chunks/bucket}
 $$
 
+Since we know that one chunk can store 4 kb of data, we can easily calculate the maximum amount of data (due to [the way batches are utilised](/docs/learn/technology/contracts/postage-stamp#batch-utilisation), postage batches may be fully utilised before storing the maximum amount of data) which can be stored by a batch from the `batch depth`:
+
+$$
+\text{Maximum data in a batch} = 2^{batchdepth} \times \text{4 kb}   
+$$
+
+For a batch of depth 16, for example, the maximum amount of data in kilobytes can be calculated like so:
+
+$$
+2^{16} \times \text{4 kb} = 65536 \text{ chunks} \times \text{4 kb} = \text{262144 kb}
+$$
 
 #### Depth to kb calculator:
 
@@ -87,12 +98,44 @@ As more chunks get uploaded and stamped, the bucket slots will begin to fill. As
 
 ### Mutable Batches
 
-Mutable batches use the same hash map structure as immutable batches, however its utilisation works very differently. In contrast with immutable batches, mutable batches are never considered fully utilised. Rather, at the point where an immutable batch would be considered fully utilised, a mutable batch can continue to stamp chunks. However, if any chunk's address lands in a bucket whose slots are already filled, that bucket's counter gets reset, and the new chunk will replace the oldest chunk in that bucket.
+Mutable batches use the same hash map structure as immutable batches, however its utilisation works very differently. In contrast with immutable batches, mutable batches are never considered fully utilised. Rather, at the point where an immutable batch would be considered fully utilised, a mutable batch can continue to stamp chunks. However, if any chunk's address lands in a bucket whose slots are already filled, rather than the batch becoming fully utilised, that bucket's counter gets reset, and the new chunk will replace the oldest chunk in that bucket.
 
 ![](/img/batches_04.png)
 
 Therefore rather than speaking of the number of slots as determining the utilisation of a batch as with immutable batches, we can think of the slots as defining a limit to the amount of data which can be uploaded before old data starts to get overwritten. 
 
+### Which Type of Batch to Use
+
+Immutable batches are suitable for long term storage of data or for data which otherwise does not need to be changed and should never be overwritten, such as records archival, legal documents, family photos, etc. 
+
+Mutable batches are great for data which needs to be frequently updated and does not require a guarantee of immutability. For example, a blog, personal or company websites, ephemeral messaging app, etc.
+
+The default batch type when unspecified is immutable. This can be modified through the Bee api by setting the `immutable` header with the [`\stamps POST` endpoint](https://docs.ethswarm.org/api/#tag/Transaction/paths/~1transactions~1%7BtxHash%7D/post) to `false`.
+
+
+### Implications for Swarm Users
+
+Due to the nature of batch utilisation described above, batches are often fully utilised before reaching their theoretical maximum storage amount. However as the batch depth increases, the chance of a postage batch becoming fully utilised early decreases. At at batch depth 23, it is statistically highly unlikely that the batch will be fully utilised/start replacing old chunks before reaching 50% of its theoretical maximum. Therefore, when choosing a batch depth, it is recommended to use a batch depth of 23 or greater, and consider 50% of its theoretical maximum storage amount as the de facto maximum storage amount. 
+
+Let's look at an example to make it clearer. Using the method of calculating the theoretical maximum storage amount [outlined above](/docs/learn/technology/contracts/postage-stamp#batch-depth), we can see that for a batch depth of 23, the maximum amount which can be stored is 33,554,432 kb:
+
+$$
+2^{23} \times \text{4kb} = \text{33,554,432 kb}
+$$
+
+Therefore we should use 50% of that value as our de facto storage limit:
+
+$$
+\text{33,554,432 kb} \times{0.5} = \text{16,777,216 kb}
+$$
+
+We find that at depth 23, we have a statistical guarantee of being able to store up to 16,777,216 kb or 16.77 gb worth of data.
+
+
+:::info
+More information about the expected utilisation levels for different batch depths will be added in the near future.
+:::
+
 
 
 

docusaurus.config.js

@@ -41,6 +41,10 @@ module.exports = {
           to: '/docs/develop/dapps-on-swarm/pss',
           from: '/docs/dapps-on-swarm/pss',
         },
+        {
+          to: '/docs/develop/access-the-swarm/buy-a-stamp-batch',
+          from: '/docs/develop/access-the-swarm/keep-your-data-alive',
+        },
       ],
     },
   ],
@@ -308,7 +312,8 @@ module.exports = {
             requiredPropsFirst: true,
             noAutoAuth: true,
             expandDefaultServerVariables: true,
-            scrollYOffset: 60
+            scrollYOffset: 60,
+            searchMaxDepth: 10
           },
         },
       },

sidebars.js

@@ -111,13 +111,13 @@ module.exports = {
       label: 'Access the Swarm',
       items: [
         'develop/access-the-swarm/introduction',
+        'develop/access-the-swarm/buy-a-stamp-batch',
         'develop/access-the-swarm/upload-and-download',
         'develop/access-the-swarm/upload-a-directory',
         'develop/access-the-swarm/direct-upload',
         'develop/access-the-swarm/store-with-encryption',
         'develop/access-the-swarm/host-your-website',
         'develop/access-the-swarm/syncing',
-        'develop/access-the-swarm/keep-your-data-alive',
         'develop/access-the-swarm/pinning',
       ],
       collapsed: false