Likely unlikely fix

[x17jiri]

RFC 1131 ( #26179 ) added likely/unlikely intrinsics, but they have been broken for a while: #96276 , #96275 , #88767 . This PR tries to fix them.

Changes:

• added a new cold_path() intrinsic
• likely() and unlikely() changed to regular functions implemented using cold_path()

[rustbot]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @cjgillot (or someone else) soon.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

• @rustbot author: the review is finished, PR author should check the comments and take action accordingly
• @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

[rustbot]

This PR changes MIR

cc @oli-obk, @RalfJung, @JakobDegen, @davidtwco, @celinval, @vakaras

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

This PR changes Stable MIR

cc @oli-obk, @celinval, @spastorino, @ouz-a

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[RalfJung]

Please make these doc comments, and also explain the last variant.

[x17jiri]

done

[RalfJung]

Why 2000?

[x17jiri]

This is what Clang emits. I will add a comment

[RalfJung]

There are similar match arms here that can be cleaned up now.

EDIT: Ah, you already did. :)

[bjorn3]

Could you please also add this at

rust/compiler/rustc_codegen_cranelift/src/base.rs

Line 817 in 69db514

NonDivergingIntrinsic::Assume(_) => {}

?

[x17jiri]

Sure. Normally, only the LLVM backend is build, right? Because I didn't get any compile time errors in GCC or Cranelift

[bjorn3]

./x.py check should check all backends. ./x.py build only builds backends other than LLVM if you tell the build system to with the rust.codegen-backends option in config.toml.

[x17jiri]

Thanks, I updated this and one more match in Cranelift

[rustbot]

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

[cjgillot]

Nit: could this be a Option<mir::ExpectKind>?

[x17jiri]

Good point. Updated

[cjgillot]

What if the Expect is not the last statement? This could happen because of MIR opts.

[x17jiri]

Could it also be moved to a different BB? Or is that something worth considering?

I was also thinking it could be useful to emit a warning if there is Expect which cannot be paired with a branch.

[x17jiri]

Updated to handle expect anywhere in a BB

[cjgillot]

For MIR opts:

• SimplifyLocals: should this statement be removed when Operand is an unused local?
• DeadStoreElimination: when the operand is a local that is never read again?
• SimplifyConstCondition: when the operand if a constant?

[x17jiri]

Opt passes updated

[cjgillot]

Nit: please split the derives in 2 lines.

[x17jiri]

Do you mean like this?

#[derive(
    Clone, TyEncodable, TyDecodable, Debug, PartialEq,
    Hash, HashStable, TypeFoldable, TypeVisitable
)]

[cjgillot]

I mean:

#[derive(Clone, TyEncodable, TyDecodable, Debug, PartialEq)]
#[derive(Hash, HashStable, TypeFoldable, TypeVisitable)]

To avoid rustfmt using a dozen lines.

[x17jiri]

Done

[x17jiri]

@rustbot ready

[rustbot]

There are merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged.

You can start a rebase with the following commands:

$ # rebase
$ git rebase -i master
$ # delete any merge commits in the editor that appears
$ git push --force-with-lease

The following commits are merge commits:

• 12a1bb3

[x17jiri]

@cjgillot I found I introduced a bug in simplify_branches.rs 🤦 It should be fixed now.

Anyway, the number of affected files is getting quite big. If there is anything I can do to make the review easier, please let me know

[Noratrieb]

small style nit (in case you're touching this code again, if you aren't it's fine to not implement it)

[Noratrieb]

Suggested change

	/// condition is probably true
	True,
	/// condition is probably false
	False,
	/// condition is unpredictable by hardware
	Unpredictable,
	/// Condition is probably true
	True,
	/// Condition is probably false
	False,
	/// Condition is unpredictable by hardware
	Unpredictable,

[bors]

☔ The latest upstream changes (presumably #120500) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

⌛ Testing commit 777003a with merge 3fb7e44...

[bors]

☀️ Test successful - checks-actions
Approved by: saethlin
Pushing 3fb7e44 to master...

[rust-timer]

Finished benchmarking commit (3fb7e44): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

• If the regression was expected or you think it can be justified,
  please write a comment with sufficient written justification, and add
  @rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
• If you think that you know of a way to resolve the regression, try to create
  a new PR with a fix for the regression.
• If you do not understand the regression or you think that it is just noise,
  you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
  were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.9%	[0.1%, 2.8%]	4
Regressions ❌ (secondary)	0.8%	[0.2%, 1.7%]	18
Improvements ✅ (primary)	-0.4%	[-1.9%, -0.1%]	21
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.1%]	11
All ❌✅ (primary)	-0.2%	[-1.9%, 2.8%]	25

Max RSS (memory usage)

Results (primary 1.1%, secondary -3.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.8%	[2.3%, 3.3%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-4.0%	[-4.0%, -4.0%]	1
Improvements ✅ (secondary)	-3.3%	[-4.0%, -2.6%]	2
All ❌✅ (primary)	1.1%	[-4.0%, 3.3%]	4

Cycles

Results (primary -0.6%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.4%	[2.4%, 2.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.3%	[-2.0%, -0.6%]	4
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.6%	[-2.0%, 2.4%]	5

Binary size

Results (primary 0.1%, secondary 0.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.0%, 1.0%]	71
Regressions ❌ (secondary)	0.5%	[0.1%, 0.7%]	78
Improvements ✅ (primary)	-0.3%	[-1.9%, -0.0%]	18
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	1
All ❌✅ (primary)	0.1%	[-1.9%, 1.0%]	89

Bootstrap: 789.328s -> 788.58s (-0.09%)
Artifact size: 335.47 MiB -> 335.03 MiB (-0.13%)

[saethlin]

@rustbot label: +perf-regression-triaged

Some change is expected because these intrinsics used to do nothing. Presumably they are used in the ecosystem, we have a few uses in the compiler too, so compiler perf here is a balance of maybe the uses in the compiler help, maybe they don't, and it's probably more work for LLVM to handle the improved IR.

Overall, strong positive on primary benchmarks and also on check builds so I do not think there is any required follow-up work. Might be fun to tune our uses of these intrinsics, I would think they're unnecessary in the compiler because we use PGO but who knows.

[klensy]

I think this fix will be nice to have in relnotes?

[RalfJung]

This is completely internal to the standard library implementation. Why would the relnotes mention it, and what would they say? We typically only document things that users need to be aware of. "Some functions are now hopefully a bit faster" doesn't go into relnotes.

[Kobzol]

If the likely/unlikely intrinsics were broken before, and now they are fixed, then that information could be quite important to some users. Both to let them know that they can now again rely on these intrinsics doing something, and also to let them know that on previous versions of Rust it probably did nothing.

Assuming that this PR is about the unstable likely/unlikely intrinsics that are available to nightly users, and thus it's not completely internal to the standard library.

[RalfJung]

Relnotes generally don't mention changes to nightly features either. They document what changed with a stable release.

[klensy]

Someone used them to tune some perf and now they see that this actually only started working as expected, so less surprises.

[RalfJung]

Given that this is on nightly now, and relnotes will be put out in ~7 weeks, it also wouldn't make a lot of sense to attach this to the Rust 1.84 relnotes.

[Kobzol]

Right, I meant the detailed release notes, which also mention internal changes and interesting pieces of information like this, not the main blog post.

[RalfJung]

How often do these detailed release notes talk about nightly features? In my understand, not very often. But maybe I misremember.

[klensy]

Personally, i think that release notes also should contain block for nightly changes, i.e. new added\removed unstable features to help users discover them, but this not related to current PR.

[RalfJung]

That would announce such changes on average 9 weeks too late. I don't think it makes a lot of sense.

[klensy]

That would announce such changes on average 9 weeks too late. I don't think it makes a lot of sense.

With release cycle of 6 weeks this will be only <= 6 weeks if placed in nearest relnotes.

How currently users can discover new features (before stabilization), excluding ones that manually track this repo? This gives lot less possible testing for that features. Or i miss something?

[RalfJung]

Such updates can and often are posted to the tracking issues, and we expect interested users to follow those.

With release cycle of 6 weeks this will be only <= 6 weeks if placed in nearest relnotes.

That would be really strange, since the codebase that is released there doesn't contain these changes.

If we want to do something like this IMO it should be a completely separate "nightly feature release notes" announcement, not part of the regular release notes. It doesn't even have to use the same 6 week cadence.