Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpc server: listen to ipv6 socket if available and --experimental-rpc-endpoint CLI option #4792

Merged
merged 75 commits into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
12c57c9
rpc: upgrade jsonrpsee v0.23
niklasad1 May 27, 2024
0aadd3d
Merge remote-tracking branch 'origin/master' into na-jsonrpsee-v0.23
niklasad1 Jun 7, 2024
15691d6
cleanup
niklasad1 Jun 7, 2024
f681f0e
make it compile
niklasad1 Jun 7, 2024
bd74645
fix test build
niklasad1 Jun 7, 2024
88a7c00
Merge remote-tracking branch 'origin/master' into na-jsonrpsee-v0.23
niklasad1 Jun 7, 2024
369db7e
jsonrpsee v0.23.1
niklasad1 Jun 10, 2024
fa8cda3
Merge remote-tracking branch 'origin/master' into na-jsonrpsee-v0.23
niklasad1 Jun 10, 2024
a2418a2
remove needless deps
niklasad1 Jun 12, 2024
5f62b6e
rpc: add support for ipv6 addr
niklasad1 Jun 13, 2024
a2e6e07
rpc: add `--rpc-listen-addr`
niklasad1 Jun 14, 2024
489a598
Merge branch 'master' into na-jsonrpsee-v0.23
niklasad1 Jun 18, 2024
f3df0da
fix bad merge
niklasad1 Jun 19, 2024
0b0cabe
add prdoc
niklasad1 Jun 19, 2024
0237951
bridges: add `serde_json` dependency
niklasad1 Jun 19, 2024
0f90c4e
fix prdoc
niklasad1 Jun 19, 2024
3115029
fix prdoc again
niklasad1 Jun 19, 2024
d969257
Merge remote-tracking branch 'origin/master' into na-jsonrpsee-v0.23
niklasad1 Jun 20, 2024
f0bd8b2
Merge remote-tracking branch 'origin/na-jsonrpsee-v0.23' into na-rpc-…
niklasad1 Jun 20, 2024
0477b67
move DenyUnsafe to rpc extension
niklasad1 Jun 21, 2024
1b4c2d3
implement niklas crazy solution
niklasad1 Jun 22, 2024
cc98347
fix nits
niklasad1 Jun 22, 2024
d1ddf14
Merge remote-tracking branch 'origin/master' into na-rpc-support-ipv6…
niklasad1 Aug 12, 2024
66b64c8
Delete prdoc/pr_4730.prdoc
niklasad1 Aug 12, 2024
a3e5f52
fix todos
niklasad1 Aug 12, 2024
0eb3e5f
sc-rpc: fix tests
niklasad1 Aug 12, 2024
11a871f
fix clippy
niklasad1 Aug 12, 2024
c564cdc
cleanup
niklasad1 Aug 13, 2024
b8e575c
remove stale tests
niklasad1 Aug 13, 2024
8b8b083
add warnings for rpc cfg
niklasad1 Aug 13, 2024
7e1c30e
fix test build
niklasad1 Aug 13, 2024
9122ba7
remove unused deps
niklasad1 Aug 13, 2024
fdd335e
fix warnings
niklasad1 Aug 13, 2024
5b1f5ff
Merge remote-tracking branch 'origin/master' into na-rpc-support-ipv6…
niklasad1 Aug 13, 2024
3e78406
fix more warnings
niklasad1 Aug 13, 2024
d08985a
rpc handler: inject deny unsafe
niklasad1 Aug 14, 2024
42bc717
update jsonrpsee
niklasad1 Aug 14, 2024
5d0afcd
update jsonrpsee
niklasad1 Aug 14, 2024
245f265
Merge remote-tracking branch 'origin/master' into na-rpc-support-ipv6…
niklasad1 Aug 14, 2024
4ea49af
update jsonrpsee
niklasad1 Aug 14, 2024
d16b0bd
remove used import
niklasad1 Aug 14, 2024
9fc691a
fix more nits
niklasad1 Aug 14, 2024
6073e86
rpc: remove slow logging
niklasad1 Aug 15, 2024
8fb416e
Merge remote-tracking branch 'origin/master' into na-rpc-support-ipv6…
niklasad1 Aug 15, 2024
7c7f8e3
rpc: allow optional ipv6 to fail
niklasad1 Aug 15, 2024
270f178
fix test build
niklasad1 Aug 15, 2024
811dfa4
fix broken tests
niklasad1 Aug 15, 2024
ce6c96f
make rpc server log backward compatible
niklasad1 Aug 15, 2024
738f435
fix some nits
niklasad1 Aug 16, 2024
e68f05f
move all rpc settings options to --rpc-endpoint
niklasad1 Aug 21, 2024
a4e01a0
fix nits
niklasad1 Aug 21, 2024
e1394e2
fix tests
niklasad1 Aug 21, 2024
32be5a2
Update substrate/client/cli/src/commands/run_cmd.rs
niklasad1 Aug 21, 2024
2ecbafd
fix grumbles
niklasad1 Aug 22, 2024
799e3e0
use proper port from the cli
niklasad1 Aug 22, 2024
d2e0844
Update substrate/client/cli/src/commands/run_cmd.rs
niklasad1 Aug 22, 2024
e452047
Merge remote-tracking branch 'origin/master' into na-rpc-support-ipv6…
niklasad1 Aug 22, 2024
2f663ea
Merge remote-tracking branch 'origin/na-rpc-support-ipv6-sockets' int…
niklasad1 Aug 22, 2024
0bb4871
Update substrate/client/cli/src/commands/run_cmd.rs
niklasad1 Aug 22, 2024
9d593c3
Delete cumulus/polkadot-parachain/src/command.rs
niklasad1 Aug 22, 2024
3827211
add prdoc and minor fixes
niklasad1 Aug 22, 2024
a4dec4e
Update prdoc/pr_4792.prdoc
niklasad1 Aug 22, 2024
9ddaf45
Update prdoc/pr_4792.prdoc
niklasad1 Aug 22, 2024
4ede846
fix toml fmt
niklasad1 Aug 22, 2024
a4ec638
Merge remote-tracking branch 'origin/na-rpc-support-ipv6-sockets' int…
niklasad1 Aug 22, 2024
829e9ca
update prdoc
niklasad1 Aug 22, 2024
c7a100e
semver v2
niklasad1 Aug 22, 2024
12e27fc
semver v3
niklasad1 Aug 23, 2024
46671b4
`--rpc-endpoint` -> `--experimental-rpc-endpoint`
niklasad1 Aug 23, 2024
0857baa
Update substrate/client/cli/src/commands/run_cmd.rs
niklasad1 Aug 23, 2024
bb8a039
Merge remote-tracking branch 'origin/master' into na-rpc-support-ipv6…
niklasad1 Aug 27, 2024
ecd1146
separate override
niklasad1 Aug 27, 2024
f490e7b
Update substrate/client/cli/src/commands/run_cmd.rs
niklasad1 Aug 28, 2024
f2d90d0
Merge branch 'master' into na-rpc-support-ipv6-sockets
niklasad1 Aug 28, 2024
304036e
cargo fmt
niklasad1 Aug 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions prdoc/pr_4792.prdoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Schema: Polkadot SDK PRDoc Schema (prdoc) v1.0.0
# See doc at https://raw.githubusercontent.com/paritytech/polkadot-sdk/master/prdoc/schema_user.json

title: "rpc: bind to `ipv6` if available and add `CLI --rpc-endpoint` to specify listen addr"
title: "rpc: bind to `ipv6` if available and add `CLI --experimental-rpc-endpoint` to specify listen addr"

doc:
- audience: Node Operator
Expand All @@ -10,16 +10,17 @@ doc:

1. The default setting is that substrate starts a rpc server that listens to localhost both ipv4 and ipv6 on the same port.
ipv6 is allowed to fail because some platforms may not support it
2. A new RPC CLI option `--rpc-endpoint` is introduced which allows to configure arbitrary listen addresses including the port,
2. A new RPC CLI option `--experimental-rpc-endpoint` is introduced which allows to configure arbitrary listen addresses including the port,
if this is enabled no other interfaces are enabled.
3. If the local addr is not found for any of the sockets the server is not started and throws an error.
4. Remove the deny_unsafe from the RPC implementations instead this is an extension to allow different polices for different interfaces/sockets
such one may enable unsafe on local interface and safe on only the external interface.
5. This new `--rpc-endpoint` has several options and in the help menu all possible parameters are documented.
5. This new `--experimental-rpc-endpoint` has several options and in the help menu all possible parameters are documented.
6. The log emitted by jsonrpc server when it has been started has been modified to indicate all started rpc endpoints.

So for instance it's now possible to start up three RPC endpoints as follows:
```
$ polkadot --rpc-endpoint "listen-addr=127.0.0.1:9944,methods=unsafe" --rpc-endpoint "listen-addr=0.0.0.0:9945,methods=safe,rate-limit=100" --rpc-endpoint "listen-addr=[::1]:9944,optional=true"
$ polkadot --experimental-rpc-endpoint "listen-addr=127.0.0.1:9944,methods=unsafe" --experimental-rpc-endpoint "listen-addr=0.0.0.0:9945,methods=safe,rate-limit=100" --experimental-rpc-endpoint "listen-addr=[::1]:9944,optional=true"
```

crates:
Expand Down
19 changes: 11 additions & 8 deletions substrate/client/cli/src/commands/run_cmd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -128,11 +128,12 @@ pub struct RunCmd {
#[arg(long, value_name = "PORT")]
pub rpc_port: Option<u16>,

/// Specify the JSON-RPC server interface and this option which can be enabled several times if
/// you want expose several RPC interfaces with different configurations.
/// EXPERIMENTAL: Specify the JSON-RPC server interface and this option which can be enabled
/// several times if you want expose several RPC interfaces with different configurations.
///
/// The format for this option is: `--rpc-endpoint"listen-addr=<ip:port>,<key=value>,..."`
/// where each option is separated by a comma and `listen-addr` is the only required param.
/// The format for this option is:
/// `--experimental-rpc-endpoint"listen-addr=<ip:port>,<key=value>,..."` where each option is
niklasad1 marked this conversation as resolved.
Show resolved Hide resolved
/// separated by a comma and `listen-addr` is the only required param.
///
/// The following options are available:
/// • listen-addr: The socket address (ip:port) to listen on. Be careful to not expose the
Expand All @@ -157,13 +158,15 @@ pub struct RunCmd {
/// • rate-limit-trust-proxy-headers: Trust proxy headers for disable rate limiting (optional)
/// • rate-limit-whitelisted-ips: Disable rate limiting for certain ip addresses (optional)
niklasad1 marked this conversation as resolved.
Show resolved Hide resolved
/// • retry-random-port: If the port is already in use, retry with a random port (optional)
///
/// Use with care, this flag is unstable and subject to change.
#[arg(
long,
num_args = 1..,
verbatim_doc_comment,
conflicts_with_all = &["rpc_external", "unsafe_rpc_external", "rpc_port", "rpc_cors", "rpc_rate_limit_trust_proxy_headers", "rpc_rate_limit", "rpc_rate_limit_whitelisted_ips", "rpc_message_buffer_capacity_per_connection", "rpc_disable_batch_requests", "rpc_max_subscriptions_per_connection", "rpc_max_request_size", "rpc_max_response_size"]
)]
pub rpc_endpoint: Vec<RpcEndpoint>,
pub experimental_rpc_endpoint: Vec<RpcEndpoint>,

/// Maximum number of RPC server connections.
#[arg(long, value_name = "COUNT", default_value_t = RPC_DEFAULT_MAX_CONNECTIONS)]
Expand Down Expand Up @@ -448,8 +451,8 @@ impl CliConfiguration for RunCmd {
}

fn rpc_addr(&self, default_listen_port: u16) -> Result<Option<Vec<RpcEndpoint>>> {
if !self.rpc_endpoint.is_empty() {
for endpoint in &self.rpc_endpoint {
if !self.experimental_rpc_endpoint.is_empty() {
for endpoint in &self.experimental_rpc_endpoint {
// Technically, `0.0.0.0` isn't a public IP address, but it's a way to listen on
// all interfaces. Thus, we consider it as a public endpoint and warn about it.
if endpoint.rpc_methods == RpcMethods::Unsafe && endpoint.is_global() ||
Copy link
Member Author

@niklasad1 niklasad1 Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

//cc @BulatSaif @PierreBesson I haven't introduced any flag such as --unsafe-external and --external here, it's up to the user to define such a thing by the listen-addr but I added a similar warning log.

Do you want extra flags for that anyway?

Now one need to do: --rpc-endpoint "listen-addr=0.0.0.0:9944,methods=unsafe"

but we could add something extra such as option for

--rpc-endpoint "listen-addr=0.0.0.0:9944,methods=unsafe,unsafe-external=true" but I don't see how that could make things better though...

Expand All @@ -462,7 +465,7 @@ impl CliConfiguration for RunCmd {
}
}

return Ok(Some(self.rpc_endpoint.clone()));
return Ok(Some(self.experimental_rpc_endpoint.clone()));
}

let (ipv4, ipv6) = rpc_interface(
Expand Down
Loading
Loading