feat: local wallets (#3976)

Co-authored-by: Hubert <[email protected]>

CHANGELOG.md

@@ -116,6 +116,12 @@ Mandatory release that includes:
 - [#3955](https://github.com/ChainSafe/forest/pull/3955) Added support for the
   NV22 _Dragon_ network upgrade, together with the required state migration.
 
+### Changed
+
+- [#3976](https://github.com/ChainSafe/forest/pull/3976) `forest-wallet`
+  defaults to using a local wallet instead of the builtin Forest wallet for
+  greater security.
+
 ### Fixed
 
 - [#4019](https://github.com/ChainSafe/forest/pull/4019) Fix Forest sending

documentation/src/SUMMARY.md

@@ -5,6 +5,7 @@
 - [Introduction](./introduction.md)
   - [Basic Usage](./basic_usage.md)
   - [Command-line interface](./cli.md)
+  - [Wallet handling](./wallet.md)
   - [Configuration](./configuration.md)
   - [Environment Variables](./environment_variables.md)
   - [Docker](./docker.md)

documentation/src/cli.md

@@ -36,100 +36,6 @@ admin token can also be set using `--token` flag:
 forest-cli --token eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJBbGxvdyI6WyJyZWFkIiwid3JpdGUiLCJzaWduIiwiYWRtaW4iXSwiZXhwIjoxNjczMjEwMTkzfQ.xxhmqtG9O3XNTIrOEB2_TWnVkq0JkqzRdw63BdosV0c <subcommand>
 ```
 
-## Sending Filecoin tokens from your wallet
-
-For sending Filecoin tokens, the Forest daemon must be running. You can do so by
-running:
-
-`forest --chain calibnet`
-
-If your wallet is empty, you can fund your wallet using this
-[faucet](https://faucet.calibration.fildev.network/funds.html). If this faucet
-is unavailable or does not work, there is an
-[alternative faucet](https://faucet.triangleplatform.com/filecoin/calibration).
-You can verify your wallet was funded after a few minutes in
-[Filscan](https://calibration.filscan.io/) by pasting the Message ID obtained
-from the faucet. Example from
-[this wallet](https://calibration.filscan.io/tipset/message-detail?cid=bafy2bzacebdverplts5qs3lwzsenzlh4rdsmvc42r6yg6suu4comr7gkbe76a).
-
-Next, send Filecoin tokens to a wallet address:
-
-`forest-cli --token <admin_token> send <wallet-address> <amount>`
-
-where `<amount>` is an amount specified in any SI units; this field is
-case-sensitive and can be specified in either short or long form (e.g., `mFIL`
-or `milliFIL`) in the range from `quetta` to `quecto`. Note that the default
-unit (if no unit is specified) is `FIL`.
-
-## Wallet
-
-Filecoin wallets are stored under the Forest data directory (e.g.,
-`~/.local/share/forest` in the case of Linux) in a `keystore` file.
-
-All wallet commands require write permissions and an admin token (`--token`) to
-interact with the keystore. The admin token can be retrieved from forest startup
-logs or by including the flag `--save-token <PATH>` during `forest` invocation.
-
-### Balance:
-
-Retrieve the FIL balance of a given address. Usage:
-`forest-wallet --token <admin_token> balance <address>`
-
-### Default:
-
-Get the default, persisted address from the keystore. Usage:
-`forest-wallet --token <admin_token> default`
-
-### Has:
-
-Check if an address exists in the keystore. Outputs `true` if address exists and
-`false` if address does not exist. Usage:
-`forest-wallet --token <admin_token> has <address>`
-
-### List:
-
-Display the keys in the keystore. Usage:
-`forest-wallet --token <admin_token> list`
-
-### New:
-
-Create a new wallet. The signature type can either be secp256k1 or bls. Defaults
-to use secp256k1. Usage:
-`forest-wallet --token <admin_token> new [ bls | secp256k1 ]`
-
-### Set-default:
-
-Set an address to be the default address of the keystore. Usage:
-`forest-wallet --token <admin_token> set-default <address>`
-
-### Export:
-
-Export a key by address. Use a wallet address to export a key. Returns a
-formatted key to be used to import on another node, or into a new keystore.
-Usage: `forest-wallet --token <admin_token> export <address>`
-
-### Import:
-
-Import a file containing a private key to the keystore and create a new address.
-The default format for importing keys is hex encoded JSON. Use the `export`
-command to get formatted keys for importing. Usage:
-`forest-wallet --token <admin_token> import <PATH>`
-
-### Sign:
-
-Use an address to sign a vector of bytes. Usage:
-`forest-wallet --token <admin_token> sign -m <hex message> -a <address>`
-
-### Verify:
-
-Verify the message's integrity with an address and signature. Outputs `true` if
-signature verifies message integrity, otherwise `false`. Usage:
-`forest-wallet verify -m <hex message> -a <address> -s <signature>`
-
-### Delete:
-
-Deletes a wallet given its address. Usage: `forest-wallet delete <address>`
-
 ## Chain-Sync
 
 The chain-sync CLI can mark blocks to never be synced, provide information about

documentation/src/wallet.md

@@ -0,0 +1,124 @@
+# Wallet handling
+
+There are two wallets for Forest: One is accessible by the Forest node, and one
+is only accessible by you. It is recommended that you only use the local wallet
+for security reasons. The wallet in the Forest node exists for backward
+compatibility with Lotus.
+
+## Configuration
+
+To query an account's balance or transfer funds, you need access to a running
+Filecoin node. You can run such a node yourself or use a publicly available
+node. As a rule of thumb, only send real money through a node that you trust.
+The rest of this document will assume you're using play money on calibnet.
+
+Glif.io runs a public Filecoin node at that we can use by setting
+`FULLNODE_API_INFO`:
+
+```bash
+export FULLNODE_API_INFO=/dns/api.calibration.node.glif.io/tcp/443/https
+```
+
+## Creating an account
+
+Initially, our wallet contains no addresses:
+
+```
+$ forest-wallet list
+Address                                   Default Balance
+```
+
+Let's create a new address and inspects its balance:
+
+```
+$ forest-wallet new
+t15ydyu3d65gznpp2qxwpkjsgz4waubeunn6upvla
+$ forest-wallet list
+Address                                   Default Balance
+t15ydyu3d65gznpp2qxwpkjsgz4waubeunn6upvla  X        0 FIL
+```
+
+The generated address will be unique and it will have a balance of `0 FIL`.
+Since this is a testnet account, we can add FIL to it from the
+[faucet](https://faucet.calibnet.chainsafe-fil.io/funds.html)/[alternate faucet](https://faucet.triangleplatform.com/filecoin/calibration).
+
+After requesting the funds and waiting roughly a minute, we can see the funds
+arrive in our wallet:
+
+```
+$ forest-wallet list
+Address                                   Default Balance
+t15ydyu3d65gznpp2qxwpkjsgz4waubeunn6upvla  X        100 FIL
+```
+
+## Sending Filecoin tokens from your wallet
+
+Let's create a new, empty account:
+
+```
+$ forest-wallet new
+t14tgmcxrcohfstxuxfbfk2vrjr3tqmefzlajp52y
+$ forest-wallet list
+Address                                   Default Balance
+t14tgmcxrcohfstxuxfbfk2vrjr3tqmefzlajp52y           0 FIL
+t15ydyu3d65gznpp2qxwpkjsgz4waubeunn6upvla  X        100 FIL
+```
+
+We can transfer FIL to this new account from our default account:
+
+```
+$ forest-wallet send t14tgmcxrcohfstxuxfbfk2vrjr3tqmefzlajp52y "1.2 FIL"
+bafy2bzaceasy7bzgjwnl4mbjp3tfxdeq4mvdvfne7fj773w7x4d6ah7cdabkc
+```
+
+It takes a minute or so for the transaction to be included in the Filecoin
+blockchain. Once the transaction has gone through, we can inspect our balances:
+
+```
+$ forest-wallet list
+Address                                   Default Balance
+t14tgmcxrcohfstxuxfbfk2vrjr3tqmefzlajp52y           1200 milliFIL
+t15ydyu3d65gznpp2qxwpkjsgz4waubeunn6upvla  X        ~98800 milliFIL
+```
+
+The gas cost of the transaction is automatically paid from the sending account.
+
+## CLI
+
+The forest-wallet executable offers several subcommand and options:
+
+```
+USAGE:
+  forest-wallet [OPTIONS] <COMMAND>
+
+SUBCOMMANDS:
+  new               Create a new wallet
+  balance           Get account balance
+  default           Get the default address of the wallet
+  export            Export the wallet's keys
+  has               Check if the wallet has a key
+  import            Import keys from existing wallet
+  list              List addresses of the wallet
+  set-default       Set the default wallet address
+  sign              Sign a message
+  validate-address  Validates whether a given string can be decoded
+                    as a well-formed address
+  verify            Verify the signature of a message. Returns true
+                    if the signature matches the message and address
+  delete            Deletes the wallet associated with the given address
+  send              Send funds between accounts
+  help              Print this message or the help of the given subcommand(s)
+
+OPTIONS:
+      --token <TOKEN>  Admin token to interact with the node
+      --remote-wallet  Use remote wallet associated with the Filecoin node
+      --encrypt        Encrypt local wallet
+  -h, --help           Print help
+  -V, --version        Print version
+```
+
+## Lotus compatiblity
+
+If you want to use the builtin wallet in a Lotus or Forest node, you can use the
+`forest-wallet` executable with the `--remote-wallet` option. The subcommands
+remain the same but require write access to the remote Filecoin node.

scripts/tests/calibnet_wallet_check.sh

@@ -29,6 +29,7 @@ forest_init
 FIL_AMT="500 atto FIL"
 
 $FOREST_WALLET_PATH import preloaded_wallet.key
+$FOREST_WALLET_PATH --remote-wallet import preloaded_wallet.key
 
 # The preloaded address
 ADDR_ONE=$($FOREST_WALLET_PATH list | tail -1 | cut -d ' ' -f1)
@@ -37,12 +38,19 @@ sleep 5s
 
 $FOREST_WALLET_PATH export "$ADDR_ONE" > preloaded_wallet.test.key
 $FOREST_WALLET_PATH delete "$ADDR_ONE"
+$FOREST_WALLET_PATH --remote-wallet delete "$ADDR_ONE"
 ROUNDTRIP_ADDR=$($FOREST_WALLET_PATH import preloaded_wallet.test.key)
 if [[ "$ADDR_ONE" != "$ROUNDTRIP_ADDR" ]]; then
     echo "Wallet address should be the same after a roundtrip"
     exit 1
 fi
 
+ROUNDTRIP_ADDR=$($FOREST_WALLET_PATH --remote-wallet import preloaded_wallet.test.key)
+if [[ "$ADDR_ONE" != "$ROUNDTRIP_ADDR" ]]; then
+    echo "Wallet address should be the same after a roundtrip"
+    exit 1
+fi
+
 wget -O metrics.log http://localhost:6116/metrics
 
 sleep 5s
@@ -55,11 +63,20 @@ ADDR_TWO=$($FOREST_WALLET_PATH new)
 echo "$ADDR_TWO"
 $FOREST_WALLET_PATH set-default "$ADDR_ONE"
 
+echo "Creating a new (remote) address to send FIL to"
+ADDR_THREE=$($FOREST_WALLET_PATH --remote-wallet new)
+echo "$ADDR_THREE"
+$FOREST_WALLET_PATH --remote-wallet set-default "$ADDR_ONE"
+
 $FOREST_WALLET_PATH list
+$FOREST_WALLET_PATH --remote-wallet list
 
-MSG=$($FOREST_CLI_PATH send "$ADDR_TWO" "$FIL_AMT")
+MSG=$($FOREST_WALLET_PATH send "$ADDR_TWO" "$FIL_AMT")
 : "$MSG"
 
+MSG_REMOTE=$($FOREST_WALLET_PATH --remote-wallet send "$ADDR_THREE" "$FIL_AMT")
+: "$MSG_REMOTE"
+
 ADDR_TWO_BALANCE=0
 i=0
 while [[ $i != 20 && $ADDR_TWO_BALANCE == 0 ]]; do
@@ -70,8 +87,19 @@ while [[ $i != 20 && $ADDR_TWO_BALANCE == 0 ]]; do
   ADDR_TWO_BALANCE=$($FOREST_WALLET_PATH balance "$ADDR_TWO")
 done
 
+ADDR_THREE_BALANCE=0
+i=0
+while [[ $i != 20 && $ADDR_THREE_BALANCE == 0 ]]; do
+  i=$((i+1))
+
+  : "Checking balance $i/20"
+  sleep 30s
+  ADDR_THREE_BALANCE=$($FOREST_WALLET_PATH --remote-wallet balance "$ADDR_TWO")
+done
+
 # wallet list should contain address two with transfered FIL amount
 $FOREST_WALLET_PATH list
+$FOREST_WALLET_PATH --remote-wallet list
 
 # wallet delete tests
 ADDR_DEL=$(forest-wallet new)
@@ -81,6 +109,14 @@ forest-wallet delete "$ADDR_DEL"
 # Validate that the wallet no longer exists.
 forest-wallet list | grep --null-data --invert-match "${ADDR_DEL}"
 
+# wallet delete tests
+ADDR_DEL=$(forest-wallet --remote-wallet new)
+
+forest-wallet --remote-wallet delete "$ADDR_DEL"
+
+# Validate that the wallet no longer exists.
+forest-wallet --remote-wallet list | grep --null-data --invert-match "${ADDR_DEL}"
+
 # TODO: Uncomment this check once the send command is fixed
 # # `$ADDR_TWO_BALANCE` is unitless (`list` command formats "500" as "500 atto FIL"),
 # # so we need to truncate units from `$FIL_AMT` for proper comparison

src/cli/subcommands/send_cmd.rs

@@ -34,6 +34,11 @@ pub struct SendCommand {
 
 impl SendCommand {
     pub async fn run(self, api: ApiInfo) -> anyhow::Result<()> {
+        eprintln!(
+            "This command has been deprecated and will be removed in the future.\n\
+             Please use the 'forest-wallet' executable instead."
+        );
+
         let from: Address =
             if let Some(from) = &self.from {
                 StrictAddress::from_str(from)?.into()

src/rpc/auth_layer.rs

@@ -64,7 +64,10 @@ static ACCESS_MAP: Lazy<HashMap<&str, Access>> = Lazy::new(|| {
     // Message Pool API
     access.insert(mpool_api::MpoolGetNonce::NAME, Access::Read);
     access.insert(mpool_api::MpoolPending::NAME, Access::Read);
-    access.insert(mpool_api::MpoolPush::NAME, Access::Write);
+    // Lotus limits `MPOOL_PUSH`` to `Access::Write`. However, since messages
+    // can always be pushed over the p2p protocol, limiting the RPC doesn't
+    // improve security.
+    access.insert(mpool_api::MpoolPush::NAME, Access::Read);
     access.insert(mpool_api::MpoolPushMessage::NAME, Access::Sign);
 
     // Sync API

src/rpc_client/gas_ops.rs

@@ -0,0 +1,32 @@
+// Copyright 2019-2024 ChainSafe Systems
+// SPDX-License-Identifier: Apache-2.0, MIT
+
+use crate::{
+    rpc::{
+        gas_api::*,
+        types::{ApiTipsetKey, MessageSendSpec},
+    },
+    shim::message::Message,
+};
+
+use super::{ApiInfo, JsonRpcError, RpcRequest};
+
+impl ApiInfo {
+    pub fn gas_estimate_message_gas_req(
+        msg: Message,
+        spec: Option<MessageSendSpec>,
+        tsk: ApiTipsetKey,
+    ) -> RpcRequest<Message> {
+        RpcRequest::new(GAS_ESTIMATE_MESSAGE_GAS, (msg, spec, tsk))
+    }
+
+    pub async fn gas_estimate_message_gas(
+        &self,
+        msg: Message,
+        spec: Option<MessageSendSpec>,
+        tsk: ApiTipsetKey,
+    ) -> Result<Message, JsonRpcError> {
+        self.call(Self::gas_estimate_message_gas_req(msg, spec, tsk))
+            .await
+    }
+}

src/rpc_client/mod.rs

@@ -5,6 +5,7 @@ pub mod beacon_ops;
 pub mod chain_ops;
 pub mod common_ops;
 pub mod eth_ops;
+pub mod gas_ops;
 pub mod net_ops;
 pub mod node_ops;
 pub mod state_ops;