document avail batcher settings (#53)

* document avail batcher settings

and other undocumented variables

* improve the docs for the avail block funnel

* reorganize the sections based on feedback

* remove delay from avail block funnel example

docs/home/1-setup/20-paima-bacher.md

@@ -49,10 +49,93 @@ First, ensure you have a config file `.env.*` ready in the root directory of you
 cat .env.mainnet >> ../.env.mainnet
 ```
 
-Of note, one key variable that needs to be set manually to use the batcher is the `BATCHER_PRIVATE_KEY`. This needs to be set as the private key of the wallet intended to be used for creating and posting the batched transactions (note, the wallet needs sufficient funds for posting to the contract). The expected format of the variable is a hex string without the `0x` prefix (ie. exactly what you get from MetaMask under Account details -> Export private key).
+## Configuration
+
+Many of the environment variables used by the batcher are shared with the
+engine, and these are already documented [here](4-environment-config-values.md).
+
+### Common settings
+
+The batcher supports submitting transactions to more than one network.  This
+section talks about settings that apply in both cases, and then follow the
+specific ones.
+
+#### Required settings
+
+Of note, one key variable that needs to be set manually to use the batcher is
+the `BATCHER_PRIVATE_KEY`.
+
+For evm based chains this needs to be set as the
+private key of the wallet intended to be used for creating and posting the
+batched transactions (note, the wallet needs sufficient funds for posting to the
+contract). The expected format of the variable is a hex string without the `0x`
+prefix (ie. exactly what you get from MetaMask under Account details -> Export
+private key).
+
+For Avail the format is that of a [Substrate
+uri](https://polkadot.js.org/docs/keyring/start/suri/).  Note this is only used
+for self signed inputs, otherwise it's handled by the light client.
+
+- `CHAIN_URI`: The URI of the chain (rpc endpoint).
+- `BATCHER_PORT`: The port to listen for submissions from the frontend.
+- `BATCHER_DB_HOST`: Host of the batcher's database.
+- `BATCHER_DB_USER`: User for the batcher's database.
+- `BATCHER_DB_PW`: Password for the batcher's database.
+- `BATCHER_DB_NAME`: Name of the database.
+- `ENABLE_RECAPTCHA_V3`: See [this section](#batcher-security-recaptcha).
+- `RECAPTCHA_V3_BACKEND`: See [this section](#batcher-security-recaptcha)
+- `BATCHED_MESSAGE_SIZE_LIMIT`: Max size limit (in characters) for batched
+transactions in a single round.
+- `MAX_USER_INPUTS_PER_MINUTE`: Per user input submission limit.
+- `MAX_USER_INPUTS_PER_DAY`: Per user input submission limit.
+- `SECURITY_NAMESPACE`: [See this section for details](../200-read-write-L2-state/600-autosign.md#defining-your-namespace).
+
+#### Optional settings
+
+- `BATCHED_TRANSACTION_POSTER_PERIOD`: Delay for the transaction poster loop
+when there are no inputs in the queue.
+- `GAME_NODE_URI`: Used for input validation.
+- `DEFAULT_VALIDATION_ACTIVE`: Whether input validation is active or not. This
+requires `GAME_NODE_URI` to also be set.
+- `GAME_INPUT_VALIDATOR_PERIOD`: Throttle for the input validator.
 
 If you plan to use the batcher in web 2.5 environment, you also need to turn on the self signing feature by setting `SELF_SIGNING_ENABLED="true"` and filling in `API_KEY` value of your choice in `SELF_SIGNING_API_KEY` variable. You'll use this key afterwards on the server communicating with the batcher.
 
+- `BATCHER_CARDANO_ENABLED_POOLS`: A comma separated list of pool credentials,
+only users delegating to one of these pools will be able to post to the batcher.
+The expected format is the public key hash (28 bytes) as a hexadecimal string
+(56 characters).
+- `CARP_URL`: The URL of a Carp instance. This is required if
+`BATCHER_CARDANO_ENABLED_POOLS` is used.
+
+### Network dependent settings
+
+- `BATCHER_NETWORK` can be used to choose the type of network the batcher submits
+to. Possible values are: `evm` and `avail`. If not set it will default to `evm`.
+
+#### Evm
+
+##### Required settings
+
+- `CONTRACT_ADDRESS`: The address of your Paima L2 contract.
+- `DEFAULT_FEE`: The fee for the contract call.
+
+##### Optional settings
+
+- `MAX_BASE_GAS`: For gas estimation. Defaults to 50000.
+- `MAX_GAS_PER_BYTE`: For gas estimation: Defaults to 32.
+
+#### Avail
+
+##### Required
+
+- `BATCHER_AVAIL_LIGHT_CLIENT`: for the Avail's light client rest api. This is
+used for the transaction poster. [Reference](https://docs.availproject.org/docs/operate-a-node/run-a-light-client/light-client-api-reference#v2submit).
+- `SECURITY_NAMESPACE`: There is no contract address in this case, so this is
+required in this case. [See this section for
+details](../200-read-write-L2-state/600-autosign.md#defining-your-namespace).
+
+
 ## Usage
 
 With all of that said and done, to compile and run the batcher using docker simply run the following in the `batcher` directory:
@@ -75,7 +158,7 @@ sh ./shutdown.sh
 
 ## Batcher Security (reCAPTCHA)
 
-As the Paima Batcher posts user submissions, you might want only to allow human users to submit data and avoid bots or malicious agents. This is a difficult task, but Paima Batcher can leverage Google's reCAPTCHA V3 and easily be integrated into games. 
+As the Paima Batcher posts user submissions, you might want only to allow human users to submit data and avoid bots or malicious agents. This is a difficult task, but Paima Batcher can leverage Google's reCAPTCHA V3 and easily be integrated into games.
 
 1. Create a reCAPTCHA V3 account and get the `site-key` and `secret-key`. (https://www.google.com/recaptcha)
   * Set `RECAPTCHA_V3_BACKEND` in the `.env.<NETWORK>` file with your `secret-key`.
@@ -84,7 +167,7 @@ As the Paima Batcher posts user submissions, you might want only to allow human
   * (OPTIONAL) Set `RECAPTCHA_V3_SCORE` in the `.env.<NETWORK>` file as a value between `0.0` and `1.0` (default `0.5` is used if not set). reCAPTCHA V3 returns a value between 0.0 and 1.0, where 0.0 is most likely a bot, and 1.0 is most likely a human. Scores < RECAPTCHA_V3_SCORE will be rejected.
 
 2. Add the reCAPTCHA code to your project
-  * Add the reCaptcha3 script `<script src="https://www.google.com/recaptcha/api.js?render=${site_key}" />` into your main HTML. 
+  * Add the reCaptcha3 script `<script src="https://www.google.com/recaptcha/api.js?render=${site_key}" />` into your main HTML.
   * Or call `injectReCaptchaToHTML()` in your frontend through the middleware.
 
 

docs/home/1-setup/4-environment-config-values.md

@@ -27,10 +27,9 @@ These these variables are used if you only need to synchronize a single network
   - `DEFAULT_PRESYNC_STEP_SIZE`: number of blocks to process in each step during initial presync phase. If not provided, a value of 1000 is used. Generally no need to change this value.
   - `CDE_CONFIG_PATH`: allows you to specify a custom location for your `extensions.yml` that is used to initialize primitive catalogue entries
 - Cardano extensions
-  - `CARP_URL`: The URL of a Carp instace, required when using Cardano primitives.
+  - `CARP_URL`: The URL of a Carp instance, required when using Cardano primitives.
   - `CARDANO_NETWORK`: One of `preview`, `preprod`, `mainnet`. It needs to match the network indexed by the Carp instance.
-  - `BATCHER_CARDANO_ENABLED_POOLS`: A comma separated list of pool credentials, only users delegating to one of these pools will be able to post to the batcher. The expected format is the public key hash (28 bytes) as a hexadecimal string (56 characters).
-- `DEFAULT_FUNNEL_GROUP_SIZE`: The number of blocks queried in one funnel sync step. If not set, a value of 100 is used. Generally no need to change this value.
+  - `DEFAULT_FUNNEL_GROUP_SIZE`: The number of blocks queried in one funnel sync step. If not set, a value of 100 is used. Generally no need to change this value.
 
 ## Others
 

docs/home/300-react-to-events/3-funnel-types/200-configuration.md

@@ -20,7 +20,7 @@ Each entry should have a `type` variable, which can be one of
 - `avail-other`
 
 There can be only one entry of the `main` type. In the case of `evm-main`, this
-network is the one that has the Paima contract deployed. 
+network is the one that has the Paima contract deployed.
 
 Currently also there should only be one entry of type `cardano` and `mina`.
 
@@ -108,6 +108,7 @@ Avail:
   confirmationDepth: 10
   funnelBlockGroupSize: 100
   presyncStepSize: 1000
+  genesisHash: "0xdd60847daa1119ecc9bdd006b140087737ac03d260ce110ecd7cb33cf721be35"
 ```
 
 ## Extensions

docs/home/300-react-to-events/3-funnel-types/700-parallel-avail-funnel.md

@@ -9,10 +9,9 @@ Avail:
   type: avail-other
   lightClient: 'http://localhost:7007'
   rpc: ws://127.0.0.1:9944
-  delay: 200 
-  confirmationDepth: 10
+  delay: 60 
   funnelBlockGroupSize: 100
-  presyncStepSize: 1000
+  genesisHash: "0xdd60847daa1119ecc9bdd006b140087737ac03d260ce110ecd7cb33cf721be35"
 ```
 
 ## Conceptually

docs/home/300-react-to-events/3-funnel-types/800-avail-block-funnel.md

@@ -1,14 +1,22 @@
 # Avail block funnel
 
+An analogous to the [evm block funnel](./300-block-funnel.md) but for Avail.
+
+This funnel will do the following:
+
+1. Get the latest block number with verified data availability from the Avail light client through the `v2/status` endpoint. [Reference](https://docs.availproject.org/docs/operate-a-node/run-a-light-client/light-client-api-reference#v2status).
+2. Fetch a group of `funnelBlockGroupSize` headers (or less if we're already at the tip).
+3. Fetch all the submitted data from the Avail light client through the `v2/blocks/{block_number}/data` endpoint ([reference](https://docs.availproject.org/docs/operate-a-node/run-a-light-client/light-client-api-reference#v2blocksblock_numberdatafieldsdataextrinsic)). This means full blocks are never processed, because the light client can filter by app id.
+4. Use `TextDecoder` to read the binary input as text, and continue with the Paima concise format processing.
+
 ## Configuration
 
 ```yaml
 Avail:
   type: avail-main
   lightClient: 'http://localhost:7007'
   rpc: ws://127.0.0.1:9944
-  delay: 200
-  confirmationDepth: 10
   funnelBlockGroupSize: 100
   presyncStepSize: 1000
+  genesisHash: "0xdd60847daa1119ecc9bdd006b140087737ac03d260ce110ecd7cb33cf721be35"
 ```