docs: major rework of trin book (#1556)

Introduction is more focused on first-time users. It pushes back some of
the conceptual stuff, tries to explain several things more clearly, and
updates some stale info.

Also, it adds an overall Portal benefits page. It's just a rough first
stab, to have something in place to update as we move forward.

book/src/SUMMARY.md

@@ -1,20 +1,23 @@
 # Summary
 
 - [Introduction](introduction/README.md)
-    - [Quickstart](introduction/quickstart.md)
-- [Users](users/README.md)
-    - [Requirements](users/requirements.md)
-    - [Startup](users/startup.md)
-    - [Use](users/use/README.md)
-        - [Making queries](users/use/making_queries.md)
+- [Using trin](introduction/quickstart.md)
+    - [Running trin](users/startup.md)
+    - [Querying Data](users/use/README.md)
+        - [Building Queries](users/use/making_queries.md)
         - [Ethereum data](users/use/ethereum_data.md)
         - [Portal network data](users/use/portal_network_data.md)
+        - [Remote access](users/use/remote_access.md)
+    - [Requirements](users/requirements.md)
     - [Monitoring](users/monitoring.md)
     - [Problems](users/problems.md)
     - [FAQ](users/faq.md)
-- [Concepts](concepts/README.md)
+    - [All Command Line Flags](users/cli.md)
+- [Portal Concepts](concepts/README.md)
     - [Portal Network](introduction/portal_network.md)
-- [Developers](developers/README.md)
+    - [Portal vs Standard Clients](concepts/portal_vs_standard.md)
+    - [Who Benefits](users/README.md)
+- [Developing trin](developers/README.md)
     - [Quick setup](developers/quick_setup.md)
     - [Developer stories](developers/developer_stories.md)
     - [Goals](developers/goals.md)

book/src/concepts/README.md

@@ -1,6 +1,6 @@
 # Concepts
 
-## Why Portal Network?
+## Why does Portal Network exist?
 
 Portal is an important way to support the evolution of the core Ethereum protocol.
 
@@ -9,7 +9,7 @@ likely future upgrade.
 
 When that happens, the Portal Network can supply users with that purged data.
 
-## Why do Portal clients use less space?
+## How do Portal clients use less space?
 
 Each Portal Network client stores a user-configurable fraction of the data. The client retrieves any missing data from peers, on demand. Just like a full node, the client can cryptographically prove the data it serves.
 

book/src/concepts/portal_vs_standard.md

@@ -0,0 +1,95 @@
+# Portal vs Standard Clients
+
+This page is not trying to be an unbiased comparison. Obviously we think Portal is cool. Below is why we think so. Of course, we still identify known drawbacks of using the Portal Network.
+
+## What are standard clients?
+
+These are also known as Execution Layer clients. The [top three by usage](https://clientdiversity.org/) are Geth, Nethermind, and Besu. There are more great ones out there.
+
+You would use standard clients if you want to stake ether, or access on-chain contracts or generate transactions without using a third-party service.
+
+## Standard client challenges
+
+In general, the following are challenges with all standard clients:
+
+- First-time sync: can take days or more
+- High storage needs: 2 Terabytes or more
+- Connectivity sensitivity: going offline overnight means spending a while to catch up
+- Non-trivial CPU usage: you'll notice it running on your laptop
+
+## Portal benefits
+
+In contrast, Portal Network was designed to overcome these challenges. For example:
+
+### Sync
+
+First-time sync is very fast. You can be up and running in minutes.
+
+All Portal needs to do is download data for the Consensus Layer to identify the tip of the chain signed by stakers. The client can then validate all past Execution Layer data.
+
+### Storage
+
+The amount you store is fully configurable. You could run with 10 MB, if you want.
+
+You will help yourself and the network by storing more, like 1-100 Gigabytes. That means you'll get faster local requests and you'll be able to serve more data to others. But it is fully your choice.
+
+### Connectivity
+
+Going offline and coming back online is no big deal. You'll be able to sync up again quickly.
+
+### CPU
+
+Portal is designed to be very light on CPU. The goal is to make it so you can run it on a Raspberry Pi, or even forget that it's running on your laptop, because it's so light.
+
+Since we're still in early days, users may experience some fluctuations in CPU usage. We will continue to optimize!
+
+## Joint Benefits of Portal & Standard Clients
+
+Some things are great about standard clients, so Portal keeps those features, like:
+
+### Standard JSON-RPC Endpoint
+
+Portal clients can act as a server for Ethereum data. They do this by hosting the standardized JSON-RPC endpoint. Portal clients are a drop-in replacement for you Web3 script or wallet.
+
+Note that not every endpoint is supported in Portal clients yet, but coverage is expanding over time.
+
+### Fully-validated
+
+Whenever Portal clients request data from a peer, they also generate internal cryptographic proofs that the provided content matches the canonical chain.
+
+This happens recursively until the source of consensus. For example, if you request contract storage, Portal clients will:
+1. generate merkle proofs back to the state root
+2. verify the state root against a header
+3. verify that the header was signed/attested by Ethereum stakers
+
+### Privacy
+
+There is no single third party that collects every request you make about the Ethereum network.
+
+An individual peer knows if you request data from them, but they don't know what your original RPC query is.
+
+## Portal Drawbacks
+
+There are some drawbacks to using Portal clients. Here are some known ones:
+
+### Latency
+
+When making a request that requires many different pieces of data under the hood, that requires many network round trips. This can be slow. Reading data out of a contract might take many seconds, instead of milliseconds.
+
+### Partial view
+
+The essense of Portal is that you only store a small slice of the total data.
+
+There are some use cases that involve seeing all the data in the network at once. For those, it will often be better to just load a standard client and have all the data locally for analysis.
+
+*Caveat:* There are still some cases that it might be faster to use Portal, even if you need a wide spread of data. You might be able to enumerate every account on the network faster than it takes a standard client to sync up from scratch, for example.
+
+### Offline access
+
+A Portal client is not very useful while offline. Clients depends on requesting missing data from peers. In contrast, standard clients that are offline can still serve all data up until their latest sync point.
+
+### Uptime
+
+The primary Ethereum network is maniacal about uptime. If you run a standard client, and have an internet connection, you will be getting network updates.
+
+There are more opportunities for downtime or lag in Portal clients. You might expect something more like 99.5% uptime. *(No promises, just a guess. Of course we will aim higher)*

book/src/introduction/README.md

@@ -4,10 +4,10 @@
 
 Trin acts as a json-rpc server, like an Ethereum client.
 
-Unlike an Ethereum client, trin has:
-- Nearly instant sync
-- Low CPU & storage usage
+Unlike a typical Ethereum client, trin:
+- is usable within minutes
+- limits storage & CPU usage
 
 Continue reading to see how to use trin.
 
-🏗 The sections, content and links of this book are subject to change.
+🏗 This is a living document, subject to substantial change without warning.

book/src/introduction/quickstart.md

@@ -2,8 +2,6 @@
 
 Trin runs on Linux, MacOS, and Windows. There are two ways to run it: download a binary executable, or install it from source.
 
-Let's download the binary executable.
-
 ## Download an executable
 
 The github repository hosts the binaries. Download the latest release for your platform from the [releases page](https://github.com/ethereum/trin/releases).
@@ -26,7 +24,7 @@ You now have a `trin` executable in the current directory.
 
 Launch the executable with 2GB local disk space:
 ```sh
-./trin --portal-subnetworks state,history --mb 2000
+trin --mb 2000
 ```
 
 ## Load a block from Ethereum history
@@ -37,10 +35,10 @@ Print the block data at height 20,987,654:
 BLOCK_NUM=20987654; echo '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x'$(printf "%x" $BLOCK_NUM)'", false],"id":1}' | nc -U /tmp/trin-jsonrpc.ipc | jq
 ```
 
-For a deeper understanding of how to interact with Ethereum, like invoking a contract function, see the [Ethereum data](/users/use/ethereum_data.md) section.
+For a deeper understanding of how to interact with Ethereum, like invoking a contract function, see the [Ethereum data](../users/use/ethereum_data.md) section.
 
 ## Alternatively, install from source
 
 To get the very latest updates, install from source. This path is intended for power users and developers, who want access to the very latest code.
 
-There are platform-specific [build instructions](/developers/contributing/build_instructions.md).
+There are platform-specific [build instructions](../developers/contributing/build_instructions.md).

book/src/users/README.md

@@ -1,6 +1,8 @@
-# Users
+# Use Cases
 
-The following are users who are well-suited to using Trin.
+The following are examples of people who will be well-suited to using a Portal Network client, like Trin.
+
+> All of these examples are speculative about the future. The most plausible users today are the [Protocol Researcher](#protocol-researcher) and [Client Developer](#client-developer).
 
 ## Laptop wallet user
 
@@ -18,13 +20,21 @@ When they want to transact, their wallet is already connected to their portal no
 *Benefit*: Wallet use without reliance on third party wallet APIs. Contributes to
 network health without using entire disk.
 
-## Protocol experimentation
+## Protocol researcher
 
 A researcher looking to explore the Ethereum protocol, testing out
 specific aspects and perhaps making experimental changes to the protocol.
 
 *Benefit*: Spin up a node and play around quickly and with low cost.
 
+## Client developer
+
+Ethereum clients are resource-intensive. Developers of those clients can update
+their client to use Portal Network data and reduce the local burden of their
+client.
+
+*Benefit*: Reduce resource usage of an Ethereum client.
+
 ## Single board computer hobbyist
 
 A raspberry pi 3, or similarly-sized computer with could contribute
@@ -39,18 +49,17 @@ network with additional robustness.
 ## Mobile user
 
 Trin is not currently configured to run on mobile, however this is plausibly
-a viable and interesting use case. A trin node could run as a background
-task with configurable limits on disk, CPU and bandwidth use.
+a viable and interesting use case for the future. There are a number of
+challenges to address first. Mobile does not typically support backrgound use
+of apps, and the battery life of a mobile device is a concern. So one challenge
+is how to make the mobile clients contribute back to the network in a way that
+is not too burdensome on the device.
 
-*Benefit*: Wallet use without reliance on third party wallet APIs. Contributes to
-network health.
+*Benefit*: Wallet use without reliance on third party wallet APIs.
 
 ## Unsuitable users
 
 There are situations where Trin is estimated to not be a good node choice:
-- Time-critical chain tip data. Likely that data distribution may not be fast enough for these
-    use cases, however testing may show otherwise.
-    - Consensus participation. Beacon chain staking with a Consensus client with Portal Network node as Execution client.
-    - Block builder. Serving blocks to beacon chain validator nodes via MEV-boost
-- Data analysis requiring state at historical blocks. Trin is not an archive node and does not
-    expose `trace_`* or` debug_`* endpoints.
+- Very speedy historical state access. It's possible to retrieve old state, but don't expect sub-second contract reads on state as viewed from a historical block.
+- Building blocks locally, as a block producer. Random access to the full state and transaction pool is not supported at the speed needed to build competitive blocks.
+  - We remain hopeful that in the future, you could use an externally-generated block (like one provided by MEV-boost) so that you can act as a validater using a standard Consensus client, with Trin as the Execution client. This probably depends on a future where state witnesses are bundled with the block sent to you by the producer.

book/src/users/cli.md

@@ -0,0 +1,67 @@
+# All Command Line Flags
+
+> See our guide on [how to run trin](startup.md) for the most common flags.
+
+Below is the current list of all command line flags.
+
+Note that these flags may change over time, so run `trin --help` to get the most up-to-date information for your version.
+
+```text
+Usage: trin [OPTIONS] [COMMAND]
+
+Commands:
+  create-dashboard  
+  help              Print this message or the help of the given subcommand(s)
+
+Options:
+      --web3-transport <WEB3_TRANSPORT>
+          select transport protocol to serve json-rpc endpoint [default: ipc]
+      --web3-http-address <WEB3_HTTP_ADDRESS>
+          address to accept json-rpc http connections [default: http://127.0.0.1:8545/]
+      --web3-ipc-path <WEB3_IPC_PATH>
+          path to json-rpc endpoint over IPC [default: /tmp/trin-jsonrpc.ipc]
+      --discovery-port <DISCOVERY_PORT>
+          The UDP port to listen on. [default: 9009]
+      --bootnodes <BOOTNODES>
+          One or more comma-delimited base64-encoded ENR's or multiaddr strings of peers to initially add to the local routing table [default: default]
+      --external-address <EXTERNAL_ADDR>
+          (Only use this if you are behind a NAT) The address which will be advertised to peers (in an ENR). Changing it does not change which port or address trin binds to. Port number is required, ex: 127.0.0.1:9001
+      --no-stun
+          Do not use STUN to determine an external IP. Leaves ENR entry for IP blank. Some users report better connections over VPN.
+      --no-upnp
+          Do not use UPnP to determine an external port.
+      --unsafe-private-key <PRIVATE_KEY>
+          Hex encoded 32 byte private key (with 0x prefix) (considered unsafe as it's stored in terminal history - keyfile support coming soon)
+      --trusted-block-root <TRUSTED_BLOCK_ROOT>
+          Hex encoded block root from a trusted checkpoint
+      --network <NETWORK>
+          Choose mainnet or angelfood [default: mainnet]
+      --portal-subnetworks <PORTAL_SUBNETWORKS>
+          Comma-separated list of which portal subnetworks to activate [default: history]
+      --storage.total <storage.total>
+          Maximum storage capacity (in megabytes), shared between enabled subnetworks [default: 1000]
+      --storage.beacon <storage.beacon>
+          Maximum storage capacity (in megabytes) used by beacon subnetwork
+      --storage.history <storage.history>
+          Maximum storage capacity (in megabytes) used by history subnetwork
+      --storage.state <storage.state>
+          Maximum storage capacity (in megabytes) used by state subnetwork
+      --enable-metrics-with-url <ENABLE_METRICS_WITH_URL>
+          Enable prometheus metrics reporting (provide local IP/Port from which your Prometheus server is configured to fetch metrics)
+      --data-dir <DATA_DIR>
+          The directory for storing application data. If used together with --ephemeral, new child directory will be created. Can be alternatively set via TRIN_DATA_PATH env variable.
+  -e, --ephemeral
+          Use new data directory, located in OS temporary directory. If used together with --data-dir, new directory will be created there instead.
+      --disable-poke
+          Disables the poke mechanism, which propagates content at the end of a successful content query. Disabling is useful for network analysis purposes.
+      --ws
+          Used to enable WebSocket rpc.
+      --ws-port <WS_PORT>
+          The WebSocket port to listen on. [default: 8546]
+      --utp-transfer-limit <UTP_TRANSFER_LIMIT>
+          The limit of max background uTP transfers for any given channel (inbound or outbound) for each subnetwork [default: 50]
+  -h, --help
+          Print help (see more with '--help')
+  -V, --version
+          Print version
+```

book/src/users/requirements.md

@@ -5,21 +5,21 @@
 Suitable:
 - Processor: x86 or Arm based. Minimum spec TBD.
 - RAM: Minimum TBD.
-- Disk: Any.
+- Disk: 50 MB
+
+We are eager to hear about a device that is too slow or small to run Trin. The minimum permitted setting for storage usage is technically 1MB. Though the trin binary itself is 41MB at the moment. Tell us if that isn't working for you!
 
 Testing and reports of performance on the following are welcome:
 - RISC-V based processor.
 - Resource constrained (CPU/RAM)
 
-
 ## Software
 
-- Unix based operating system
-- Rust installation (minimum `v1.66`)
+- Linux, MacOS, or Windows
 
 ## Network
 
 Testing/reports of low-bandwidth network are welcome.
 
 Trin should be compatible with VPN use, but if you experience difficulty
-connecting to the network we recommend disabling your VPN.
+connecting to the network we recommend disabling your VPN.