Skip to content

Commit

Permalink
Support incompatible enable proto toolchain resolution (#2741)
Browse files Browse the repository at this point in the history
This allows using a shared (generally pre-built) protoc definition,
rather than needing to supply one via a label.

We allow (but deprecate) users to still bring along their own protoc as
a label, but if they want to do so, they will need to ensure they have
the correct repositories registered in order to build/bring their
protoc, rather than us setting them up by default.
  • Loading branch information
illicitonion authored Jul 19, 2024
1 parent ca7a3df commit ffe8eff
Show file tree
Hide file tree
Showing 49 changed files with 1,091 additions and 49 deletions.
6 changes: 6 additions & 0 deletions .bazelci/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -765,6 +765,12 @@ tasks:
working_directory: examples/bzlmod/proto
build_targets:
- "//..."
bzlmod_proto:
name: Proto and Prost with prebuilt protoc with bzlmod
platform: ubuntu2004
working_directory: examples/bzlmod/proto_with_toolchain
build_targets:
- "//..."
compile_one_dependency:
name: --compile_one_dependency flag
platform: ubuntu2004
Expand Down
3 changes: 1 addition & 2 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ bazel_dep(
)
bazel_dep(
name = "rules_proto",
version = "5.3.0-21.7",
version = "6.0.2",
)
bazel_dep(
name = "apple_support",
Expand Down Expand Up @@ -68,7 +68,6 @@ use_repo(
"cargo_bazel.buildifier-linux-amd64",
"cargo_bazel.buildifier-linux-arm64",
"cargo_bazel.buildifier-windows-amd64.exe",
"com_google_googleapis",
"cui",
"cui__anyhow-1.0.75",
"cui__camino-1.1.6",
Expand Down
1 change: 1 addition & 0 deletions docs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ bzl_library(
"@com_google_protobuf//:bzl_srcs",
],
deps = [
"@bazel_features//:deps",
"@bazel_features//:features",
"@bazel_skylib//lib:paths",
"@bazel_skylib//lib:selects",
Expand Down
2 changes: 1 addition & 1 deletion docs/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ bazel_dep(
)
bazel_dep(
name = "bazel_features",
version = "1.11.0",
version = "1.14.0",
)
10 changes: 7 additions & 3 deletions docs/flatten.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,7 +809,7 @@ Rust Prost toolchain rule.
| <a id="rust_prost_toolchain-prost_plugin_flag"></a>prost_plugin_flag | Prost plugin flag format. (e.g. `--plugin=protoc-gen-prost=%s`) | String | optional | `"--plugin=protoc-gen-prost=%s"` |
| <a id="rust_prost_toolchain-prost_runtime"></a>prost_runtime | The Prost runtime crates to use. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="rust_prost_toolchain-prost_types"></a>prost_types | The Prost types crates to use. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="rust_prost_toolchain-proto_compiler"></a>proto_compiler | The protoc compiler to use. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="rust_prost_toolchain-proto_compiler"></a>proto_compiler | The protoc compiler to use. Note that this attribute is deprecated - prefer to use --incompatible_enable_proto_toolchain_resolution. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="rust_prost_toolchain-tonic_opts"></a>tonic_opts | Additional options to add to Tonic. | List of strings | optional | `[]` |
| <a id="rust_prost_toolchain-tonic_plugin"></a>tonic_plugin | Additional plugins to add to Tonic. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="rust_prost_toolchain-tonic_plugin_flag"></a>tonic_plugin_flag | Tonic plugin flag format. (e.g. `--plugin=protoc-gen-tonic=%s`)) | String | optional | `"--plugin=protoc-gen-tonic=%s"` |
Expand Down Expand Up @@ -1778,14 +1778,18 @@ This macro should be called immediately after the `rust_protobuf_dependencies` m
rust_proto_protobuf_dependencies(<a href="#rust_proto_protobuf_dependencies-bzlmod">bzlmod</a>)
</pre>


Sets up dependencies for rules_rust's proto support.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_proto_protobuf_dependencies-bzlmod"></a>bzlmod | <p align="center"> - </p> | `False` |
| <a id="rust_proto_protobuf_dependencies-bzlmod"></a>bzlmod | Whether this function is being called from a bzlmod context rather than a workspace context. | `False` |

**RETURNS**

A list of structs containing information about root module deps to report to bzlmod's extension_metadata.


<a id="rust_proto_protobuf_register_toolchains"></a>
Expand Down
10 changes: 7 additions & 3 deletions docs/rust_proto.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ Rust Prost toolchain rule.
| <a id="rust_prost_toolchain-prost_plugin_flag"></a>prost_plugin_flag | Prost plugin flag format. (e.g. `--plugin=protoc-gen-prost=%s`) | String | optional | `"--plugin=protoc-gen-prost=%s"` |
| <a id="rust_prost_toolchain-prost_runtime"></a>prost_runtime | The Prost runtime crates to use. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="rust_prost_toolchain-prost_types"></a>prost_types | The Prost types crates to use. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="rust_prost_toolchain-proto_compiler"></a>proto_compiler | The protoc compiler to use. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="rust_prost_toolchain-proto_compiler"></a>proto_compiler | The protoc compiler to use. Note that this attribute is deprecated - prefer to use --incompatible_enable_proto_toolchain_resolution. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="rust_prost_toolchain-tonic_opts"></a>tonic_opts | Additional options to add to Tonic. | List of strings | optional | `[]` |
| <a id="rust_prost_toolchain-tonic_plugin"></a>tonic_plugin | Additional plugins to add to Tonic. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="rust_prost_toolchain-tonic_plugin_flag"></a>tonic_plugin_flag | Tonic plugin flag format. (e.g. `--plugin=protoc-gen-tonic=%s`)) | String | optional | `"--plugin=protoc-gen-tonic=%s"` |
Expand Down Expand Up @@ -434,14 +434,18 @@ This macro should be called immediately after the `rust_protobuf_dependencies` m
rust_proto_protobuf_dependencies(<a href="#rust_proto_protobuf_dependencies-bzlmod">bzlmod</a>)
</pre>


Sets up dependencies for rules_rust's proto support.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_proto_protobuf_dependencies-bzlmod"></a>bzlmod | <p align="center"> - </p> | `False` |
| <a id="rust_proto_protobuf_dependencies-bzlmod"></a>bzlmod | Whether this function is being called from a bzlmod context rather than a workspace context. | `False` |

**RETURNS**

A list of structs containing information about root module deps to report to bzlmod's extension_metadata.


<a id="rust_proto_protobuf_register_toolchains"></a>
Expand Down
4 changes: 4 additions & 0 deletions examples/bzlmod/all_deps_vendor/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ bazel_dep(name = "bazel_skylib", version = "1.7.1")

# https://github.com/bazelbuild/rules_rust/releases
bazel_dep(name = "rules_rust", version = "0.46.0")
local_path_override(
module_name = "rules_rust",
path = "../../..",
)

###############################################################################
# T O O L C H A I N S
Expand Down
4 changes: 4 additions & 0 deletions examples/bzlmod/compile_opt/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ module(
###############################################################################
# https://github.com/bazelbuild/rules_rust/releases
bazel_dep(name = "rules_rust", version = "0.46.0")
local_path_override(
module_name = "rules_rust",
path = "../../..",
)

###############################################################################
# T O O L C H A I N S
Expand Down
4 changes: 4 additions & 0 deletions examples/bzlmod/cross_compile/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ module(

# https://github.com/bazelbuild/rules_rust/releases
bazel_dep(name = "rules_rust", version = "0.46.0")
local_path_override(
module_name = "rules_rust",
path = "../../..",
)

# Rules for cross compilation
bazel_dep(name = "toolchains_musl", version = "0.1.16", dev_dependency = True)
Expand Down
4 changes: 4 additions & 0 deletions examples/bzlmod/ffi/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ module(
###############################################################################
# https://github.com/bazelbuild/rules_rust/releases
bazel_dep(name = "rules_rust", version = "0.46.0")
local_path_override(
module_name = "rules_rust",
path = "../../..",
)

###############################################################################
# T O O L C H A I N S
Expand Down
4 changes: 4 additions & 0 deletions examples/bzlmod/proto/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ module(
###############################################################################
# https://github.com/bazelbuild/rules_rust/releases
bazel_dep(name = "rules_rust", version = "0.46.0")
local_path_override(
module_name = "rules_rust",
path = "../../..",
)

#
# Rules for protobuf / gRPC
Expand Down
1 change: 0 additions & 1 deletion examples/bzlmod/proto/build/prost_toolchain/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ rust_prost_toolchain(
prost_plugin = "@crates//:protoc-gen-prost__protoc-gen-prost",
prost_runtime = ":prost_runtime",
prost_types = "@crates//:prost-types",
proto_compiler = "@protobuf//:protoc",
tonic_plugin = "@crates//:protoc-gen-tonic__protoc-gen-tonic",
tonic_runtime = ":tonic_runtime",
)
Expand Down
9 changes: 9 additions & 0 deletions examples/bzlmod/proto_with_toolchain/.bazelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Required on windows
common --enable_platform_specific_config
startup --windows_enable_symlinks
build:windows --enable_runfiles

# Required for cargo_build_script support before Bazel 7
build --incompatible_merge_fixed_and_default_shell_env

common --incompatible_enable_proto_toolchain_resolution
1 change: 1 addition & 0 deletions examples/bzlmod/proto_with_toolchain/.bazelversion
5 changes: 5 additions & 0 deletions examples/bzlmod/proto_with_toolchain/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/bazel-*
.DS_Store
/proto/.DS_Store
/proto/grpc_server/.DS_Store
/proto/proto_bindings/.DS_Store
6 changes: 6 additions & 0 deletions examples/bzlmod/proto_with_toolchain/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
config_setting(
name = "release",
values = {
"compilation_mode": "opt",
},
)
42 changes: 42 additions & 0 deletions examples/bzlmod/proto_with_toolchain/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
[workspace]
resolver = "2"

members = [
"proto_bindings",
"grpc_server",
"grpc_client",
]


[workspace.package]
edition = "2021"
rust-version = "1.78.0"
readme = "README.md"


[workspace.dependencies]
# Internal crates
proto_bindings = { path = "proto_bindings" }
# External crates
prost = { version = "0.12.6" }
prost-types = { version = "0.12.6", default-features = false }
tonic = { version = "0.11.0", features = ["transport"] }
tonic-build = "0.11.0"
tokio = { version = "1.38", default-features = false, features = ["macros", "net", "rt-multi-thread", "signal"] }


# Optimize all crates
[profile.release]
opt-level = 3
strip = true # Automatically strip debug symbols from the binary
lto = true # Enable Link Time Optimization (LTO)
codegen-units = 1 # Reduce Parallel Code Generation Units to Increase Optimization


# There's a Cargo feature named profile-overrides
# that lets you override the optimization level of dependencies.
# https://docs.rust-embedded.org/book/unsorted/speed-vs-size.html
[profile.release.package."*"]
opt-level = 3
strip = 'debuginfo' # Automatically strip debug infos from the binary to reduce size
codegen-units = 1 # Reduce Parallel Code Generation Units to Increase Optimization
133 changes: 133 additions & 0 deletions examples/bzlmod/proto_with_toolchain/MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
module(
name = "grpc-client-server",
version = "0.0.0",
)

###############################################################################
# B A Z E L C E N T R A L R E G I S T R Y # https://registry.bazel.build/
###############################################################################
# https://github.com/bazelbuild/rules_rust/releases
bazel_dep(name = "rules_rust", version = "0.46.0")
local_path_override(
module_name = "rules_rust",
path = "../../..",
)

#
# Rules for protobuf / gRPC
# https://github.com/bazelbuild/rules_proto/releases
bazel_dep(name = "rules_proto", version = "6.0.2")

# https://github.com/aspect-build/toolchains_protoc/releases
bazel_dep(name = "toolchains_protoc", version = "0.3.1")

# https://registry.bazel.build/modules/protobuf
bazel_dep(name = "protobuf", version = "27.1")

# https://github.com/bazel-contrib/toolchains_llvm
bazel_dep(name = "toolchains_llvm", version = "1.0.0")

###############################################################################
# L L V M
# https://github.com/bazel-contrib/toolchains_llvm/blob/master/tests/MODULE.bazel
###############################################################################
llvm = use_extension("@toolchains_llvm//toolchain/extensions:llvm.bzl", "llvm")

# LLVM Versions and platforms
# https://github.com/bazel-contrib/toolchains_llvm/blob/master/toolchain/internal/llvm_distributions.bzl
LLVM_VERSIONS = {
"": "16.0.0",
"darwin-aarch64": "16.0.3",
"darwin-x86_64": "15.0.7",
}

# Host LLVM toolchain.
llvm.toolchain(
name = "llvm_toolchain",
llvm_versions = LLVM_VERSIONS,
)
use_repo(llvm, "llvm_toolchain", "llvm_toolchain_llvm")

register_toolchains("@llvm_toolchain//:all")

###############################################################################
# T O O L C H A I N S
###############################################################################

# Rust toolchain
RUST_EDITION = "2021"

RUST_VERSION = "1.79.0"

rust = use_extension("@rules_rust//rust:extensions.bzl", "rust")
rust.toolchain(
edition = RUST_EDITION,
versions = [RUST_VERSION],
)
use_repo(rust, "rust_toolchains")

register_toolchains("@rust_toolchains//:all")

# Proto toolchain
register_toolchains("@rules_rust//proto/protobuf:default-proto-toolchain")

# Custom Prost toolchain
register_toolchains("@//build/prost_toolchain")

###############################################################################
# R U S T C R A T E S
###############################################################################
crate = use_extension("@rules_rust//crate_universe:extension.bzl", "crate")

#
# protobuf / gRPC dependencies
crate.spec(
package = "prost",
version = "0.12",
)
crate.spec(
default_features = False,
package = "prost-types",
version = "0.12",
)
crate.spec(
features = ["transport"],
package = "tonic",
version = "0.11",
)
crate.spec(
package = "tonic-build",
version = "0.11",
)
crate.spec(
package = "protoc-gen-prost",
version = "0.3.1",
)
crate.annotation(
crate = "protoc-gen-prost",
gen_binaries = ["protoc-gen-prost"],
)
crate.spec(
package = "protoc-gen-tonic",
version = "0.4.0",
)
crate.annotation(
crate = "protoc-gen-tonic",
gen_binaries = ["protoc-gen-tonic"],
)

#
# External crates
crate.spec(
default_features = False,
features = [
"macros",
"net",
"rt-multi-thread",
"signal",
],
package = "tokio",
version = "1.38",
)
crate.from_specs()
use_repo(crate, "crates")
Loading

0 comments on commit ffe8eff

Please sign in to comment.