Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor third-party-dependencies for bzlmod #314

Merged
merged 1 commit into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions third-party-dependencies/.bazelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
common --cxxopt=-std=c++14
1 change: 1 addition & 0 deletions third-party-dependencies/.bazelversion
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6.3.2
4 changes: 2 additions & 2 deletions third-party-dependencies/BUILD
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
load("@com_github_bazelbuild_buildtools//buildifier:def.bzl", "buildifier")
load("@buildifier_prebuilt//:rules.bzl", "buildifier")
load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_library", "cc_test")

package(default_visibility = ["//visibility:public"])
Expand Down Expand Up @@ -28,7 +28,7 @@ cc_test(
deps = [
":my_lib",
"@catch2",
"@catch2//:catch2_with_main",
"@catch2//:catch2_main",
],
)

Expand Down
7 changes: 7 additions & 0 deletions third-party-dependencies/MODULE.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"Declares the third-party dependencies when bzlmod is enabled"

bazel_dep(name = "buildifier_prebuilt", version = "6.3.3", dev_dependency = True)

bazel_dep(name = "rules_cc", version = "0.0.9")
bazel_dep(name = "catch2", version = "3.4.0")
bazel_dep(name = "googletest", version = "1.14.0", repo_name = "gtest")
62 changes: 62 additions & 0 deletions third-party-dependencies/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
Bazel with third party dependencies
===================================

Goal
----

This is a small Bazel example that uses third party dependencies ([also called external dependencies](https://docs.bazel.build/versions/master/external.html))
but is well structured following the principles of software development. The main goal is that the principles and guidelines shown here they should scale big.
Bazel consultants with access to over 40 codebases have seen this pattern deployed successfully at scale.

Because of this, we want to keep the following requirements:

* Works with bzlmod both enabled and disabled, as a repository may need during a migration.
* We don't want all dependencies listed in the `WORKSPACE` file. It mixes order-dependent and order-independent code, and makes the file harder to manage. Thus we forbid `http_archive` and other "fetch" functions from being loaded in `WORKSPACE`
* The principles mentioned here should be applicable to any programming language.

Details (bzlmod)
----------------

The `MODULE.bazel` file lists the dependencies. When bzlmod is enabled, the `WORKSPACE.bzlmod` file
takes priority over the `WORKSPACE` file. Since `WORKSPACE.bzlmod` is empty, it means this
workspace has been fully converted to work with bzlmod.

Details (WORKSPACE)
-------------------
All dependencies need to be loaded in the workspace file but this does not prevent us on splitting this in several files. The first split comes on the [`WORKSPACE`](WORKSPACE), first we load the source code of all declared dependencies by calling `fetch_deps`, and then we finish what is left to have a fully loaded dependency.

Because WORKSPACE doesn't have the transitive dependency semantics of bzlmod, it requires us as the user to repeat the transitive dependencies. After declaring our external deps to fetch, we must
load and then call the transitive dependency fetching function from each direct dependency, and then
any initializers such as toolchain registration. The rules should indicate what WORKSPACE incantation
is required in their README or release notes.

In cases where the versions of some transitive dependency are skewed, it can be hard to tell how this
relates to the order of the WORKSPACE calls. In this case, you can add transitive dependencies to
the `fetch_deps` macro as a workaround. Since bzlmod will be the only way to specify dependencies
in some future version of Bazel, you can treat this as a short-term problem.

## How to try it out

In this example you can find a C++ binary, a C++ library and two C++ tests, one using [Catch2](https://github.com/catchorg/Catch2) and the other one using [Google Test](https://github.com/google/googletest).

You can run the C++ binary with the following command:

```bash
bazel run //:hello_world
```

You can run the tests with the following command:

```bash
bazel test //:all --test_output=streamed
```

For the Catch2 test, you can add additinal parameters for a nicer output:

```bash
bazel test //:catch2_test --test_arg "--reporter compact" --test_arg --success --test_output=streamed
```

If you want to know more about the parameters of the bazel command, you can check [here](https://docs.bazel.build/versions/master/command-line-reference.html)

To try with bzlmod, add the `--enable_bzlmod` flag to any of the bazel commands above.
70 changes: 0 additions & 70 deletions third-party-dependencies/Readme.md

This file was deleted.

47 changes: 27 additions & 20 deletions third-party-dependencies/WORKSPACE
Original file line number Diff line number Diff line change
@@ -1,28 +1,35 @@
# An example of the common pattern for loading dependencies.
#
# We cannot fully load a dependency without loading its transitive dependencies.
# This set of dependencies could be large and change over time, so we do not
# want to manage it in our own WORKSPACE. The pattern we use is:
#
# - Rule sets declare a method to load their dependencies.
# - We load that method and call it.
# - This pattern, of course, is repeated at each level.
# When bzlmod is enabled, this file is unused; WORKSPACE.bzlmod is used instead.
workspace(name = "bazel_example_3p")

# Fetches are order-independent, so we declare those first.
# All http_archive, http_file, etc. rules should be in the repositories.bzl file.
# DO NOT load those functions here in WORKSPACE.
load("//third_party:repositories.bzl", "fetch_deps")

fetch_deps()

# The remainder of this file is highly order-dependent. As much as possible, the fetch_deps
# call above has already pinned our dependencies to the exact version we want.
#
# We also want some fine grained control of diamond dependencies. That is,
# when we have a direct dependency on a rule set, but it is also a transitive
# dependency of one of our dependencies, we would like to make sure we always
# get the version we specify, rather than the one another rule set might.
# To ensure that, we load the 3rd party dependencies in several phases.
# If possible, migrate to bzlmod rather than struggle with the implications of moving lines
# and getting different dependency verisons.
#
# First, we import the all rules we depend on directly.
# Then, for each of our dependencies
# For each of our direct dependencies:
# - load and run their dependency loader method.
# - register any toolchains they might provide which we intend to use.

load("//third_party:third_party.bzl", "load_third_party_libraries")
load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")

bazel_skylib_workspace()

load("@buildifier_prebuilt//:deps.bzl", "buildifier_prebuilt_deps")

buildifier_prebuilt_deps()

load("@buildifier_prebuilt//:defs.bzl", "buildifier_prebuilt_register_toolchains")

load_third_party_libraries()
buildifier_prebuilt_register_toolchains()

load("//third_party:transitive_dependencies.bzl", "load_transitive_dependencies")
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")

load_transitive_dependencies()
protobuf_deps()
1 change: 1 addition & 0 deletions third-party-dependencies/WORKSPACE.bzlmod
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# This file replaces `WORKSPACE` when --enable_bzlmod is set.
2 changes: 1 addition & 1 deletion third-party-dependencies/catch2_test.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#define CATCH_CONFIG_MAIN

#include "my_lib.h"
#include <catch2/catch.hpp>
#include <catch2/catch_test_macros.hpp>

TEST_CASE( "Sum negative values", "[sum]" ) {
REQUIRE( my_lib::sum(-2, -5) == -7 );
Expand Down
Empty file.
17 changes: 0 additions & 17 deletions third-party-dependencies/third_party/bazel_skylib/direct.bzl

This file was deleted.

This file was deleted.

Empty file.
27 changes: 0 additions & 27 deletions third-party-dependencies/third_party/buildtools/direct.bzl

This file was deleted.

12 changes: 0 additions & 12 deletions third-party-dependencies/third_party/buildtools/transitive.bzl

This file was deleted.

Empty file.
27 changes: 0 additions & 27 deletions third-party-dependencies/third_party/catch2/direct.bzl

This file was deleted.

8 changes: 0 additions & 8 deletions third-party-dependencies/third_party/catch2/transitive.bzl

This file was deleted.

Empty file.
27 changes: 0 additions & 27 deletions third-party-dependencies/third_party/gtest/direct.bzl

This file was deleted.

8 changes: 0 additions & 8 deletions third-party-dependencies/third_party/gtest/transitive.bzl

This file was deleted.

Empty file.
Loading
Loading