Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add 'manual' option for //rust/settings/lto #3120

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix: add 'manual' option for //rust/settings/lto #3120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
07793a6
start, add 'manual' option for rust/settings/lto
ParkMyCar Dec 18, 2024
7e293d9
cleanup implementation
ParkMyCar Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions rust/private/lto.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
load("//rust/private:utils.bzl", "is_exec_configuration")

_LTO_MODES = [
# Do nothing, let the user manually handle LTO.
"manual",
# Default. No mode has been explicitly set, rustc will do "thin local" LTO
# between the codegen units of a single crate.
"unspecified",
Expand Down Expand Up @@ -64,8 +66,6 @@ def _determine_lto_object_format(ctx, toolchain, crate_info):
return "only_object"

perform_linking = crate_info.type in ["bin", "staticlib", "cdylib"]

# is_linkable = crate_info.type in ["lib", "rlib", "dylib", "proc-macro"]
is_dynamic = crate_info.type in ["dylib", "cdylib", "proc-macro"]
needs_object = perform_linking or is_dynamic

Expand Down Expand Up @@ -94,17 +94,21 @@ def construct_lto_arguments(ctx, toolchain, crate_info):
list: A list of strings that are valid flags for 'rustc'.
"""
mode = toolchain._lto.mode
format = _determine_lto_object_format(ctx, toolchain, crate_info)

# The user is handling LTO on their own, don't add any arguments.
if mode == "manual":
Copy link

@havasd havasd Dec 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we go with this approach, we should fix the conditions around line 110 and 113

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can definitely iterate! Are you referring to the elif format == "only_bitcode" condition?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was referring to these which are not working today correctly. If we choose this approach we should fix the below conditions according to the new behavior.

    if format in ["unspecified", "object_and_bitcode"]:
        # Embedding LLVM bitcode in object files is `rustc's` default.
        args.extend([])
    elif format in ["off", "only_object"]:
        args.extend(["embed-bitcode=no"])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet, more than happy to change these if you can describe how they're broken? If you can share an error message you're getting from a build that would be great!

I ask because I've tested all variants of the LTO setting on a relatively large project that links a few large C libraries (e.g. librdkafka, rocksdb, openssl) and all of the builds succeed?

Copy link

@havasd havasd Dec 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean that they logically wrong. (enable embed-bitcode=no for off and unspecified, don't set anything for object_and_bitcode format)

Based on what you are proposing it should look something like this in my opinion:

    if format == "object_and_bitcode":
        # Embedding LLVM bitcode in object files is `rustc's` default.
        args.extend([])
    elif mode in ["unspecified", "off"] or format == "only_object":
        args.extend(["embed-bitcode=no"])

Copy link
Contributor Author

@ParkMyCar ParkMyCar Dec 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks for pointing this out!

As-is the existing implementation is logically the same as you've written here. I think where the disconnect comes from is how "unspecified" and "off" are still referenced w.r.t. format. In a previous iteration of the original PR they were formats but I realized that was confusing, so I changed _determine_lto_object_format to return one of "only_object", "only_bitcode", or "object_and_bitcode". "unspecified" and "off" already map to "only_object" but making this clearer is a good call!

return []

format = _determine_lto_object_format(ctx, toolchain, crate_info)
args = []

if mode in ["thin", "fat", "off"] and not is_exec_configuration(ctx):
args.append("lto={}".format(mode))

if format in ["unspecified", "object_and_bitcode"]:
if format == "object_and_bitcode":
# Embedding LLVM bitcode in object files is `rustc's` default.
args.extend([])
elif format in ["off", "only_object"]:
elif format == "only_object":
args.extend(["embed-bitcode=no"])
elif format == "only_bitcode":
args.extend(["linker-plugin-lto"])
Expand Down
14 changes: 14 additions & 0 deletions test/unit/lto/lto_test_suite.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@ _lto_level_default_test = analysistest.make(
config_settings = {},
)

def _lto_level_manual(ctx):
return _lto_test_impl(ctx, None, None, False)

_lto_level_manual_test = analysistest.make(
_lto_level_manual,
config_settings = {str(Label("//rust/settings:lto")): "manual"},
)

def _lto_level_off(ctx):
return _lto_test_impl(ctx, "off", "no", False)

Expand Down Expand Up @@ -97,6 +105,11 @@ def lto_test_suite(name):
target_under_test = ":lib",
)

_lto_level_manual_test(
name = "lto_level_manual_test",
target_under_test = ":lib",
)

_lto_level_off_test(
name = "lto_level_off_test",
target_under_test = ":lib",
Expand All @@ -116,6 +129,7 @@ def lto_test_suite(name):
name = name,
tests = [
":lto_level_default_test",
":lto_level_manual_test",
":lto_level_off_test",
":lto_level_thin_test",
":lto_level_fat_test",
Expand Down
Loading