diff --git a/triage/2024-11-19.md b/triage/2024-11-19.md new file mode 100644 index 000000000..34ba79d53 --- /dev/null +++ b/triage/2024-11-19.md @@ -0,0 +1,268 @@ +# 2024-11-19 Triage Log + +We saw improvements to a large swath of benchmarks with the querification of +MonoItem collection (PR #132566). There were also some PRs where we are willing +to pay a compile-time cost for expected runtime benefit (PR #132870, PR #120370), +or pay a small cost in the single-threaded case in exchange for a big parallel +compilation win (PR #124780). + +Triage done by **@pnkfelix**. +Revision range: [d4822c2d..7d40450b](https://perf.rust-lang.org/?start=d4822c2d84c242cc7403118b50c571464f38ef8f&end=7d40450b2df92bdc9dec414b30cf5f7a5979a92e&absolute=false&stat=instructions%3Au) + +**Summary**: + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:---------------:|:-----:| +| Regressions ❌
(primary) | 1.1% | [0.2%, 7.3%] | 195 | +| Regressions ❌
(secondary) | 1.3% | [0.1%, 5.0%] | 194 | +| Improvements ✅
(primary) | -7.3% | [-24.9%, -0.3%] | 60 | +| Improvements ✅
(secondary) | -9.4% | [-67.0%, -0.3%] | 23 | +| All ❌✅ (primary) | -0.9% | [-24.9%, 7.3%] | 255 | + + +2 Regressions, 4 Improvements, 10 Mixed; 6 of them in rollups +47 artifact comparisons made in total + +#### Regressions + +Rollup of 5 pull requests [#133039](https://github.com/rust-lang/rust/pull/133039) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=c82e0dff84f922274c3060f09b5ae78d5dbf5c49&end=90ab8eaedd3efd7919bffa7638db66e67df3f0b8&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:----:|:------------:|:-----:| +| Regressions ❌
(primary) | 0.3% | [0.3%, 0.3%] | 1 | +| Regressions ❌
(secondary) | 0.3% | [0.2%, 0.4%] | 13 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | 0.3% | [0.3%, 0.3%] | 1 | + +* sole primary regression is to doc (full) on helloworld. +* already triaged as being due to PR #133005 (and categorized as "improvements to rustdoc search that justify this cost") + +Rollup of 7 pull requests [#133120](https://github.com/rust-lang/rust/pull/133120) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=ee4a56e353dc3ddfcb12df5fe2dc1329a315c2f5&end=1e0df74445a185fc97b7e806cd6c636eaded1326&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:----:|:------------:|:-----:| +| Regressions ❌
(primary) | 0.2% | [0.1%, 0.3%] | 25 | +| Regressions ❌
(secondary) | 0.2% | [0.1%, 0.4%] | 14 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | 0.2% | [0.1%, 0.3%] | 25 | + +* primary regressions to {serde, libc, diesel, cargo, cranelift-codegen} (across multiple scenarios) and {html5ever, ripgrep, exa, syn, stm32f4, image, regex, webrender, hyper} (in isolated scenarios). +* @nnethercote has hypothesized that PR #132134 may be responsible and is planning to investigate. +* Not marking as triaged. + +#### Improvements + +move all mono-time checks into their own folder, and their own query [#132843](https://github.com/rust-lang/rust/pull/132843) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=5700240affd222f69b8755e2ff5d4ccaae9e6cf9&end=583b25d8d1bf934f593d9d9811f88305888032b5&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.2% | [-0.2%, -0.2%] | 4 | +| Improvements ✅
(secondary) | -0.4% | [-0.4%, -0.4%] | 1 | +| All ❌✅ (primary) | -0.2% | [-0.2%, -0.2%] | 4 | + +* (based on conversation in the PR itself, I think this "gain" is illusory and these benchmarks are just unstable) + +Rollup of 4 pull requests [#133047](https://github.com/rust-lang/rust/pull/133047) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=90ab8eaedd3efd7919bffa7638db66e67df3f0b8&end=e84902d35a4d3039c794e139eb12fba3624c5ff1&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.2% | [-0.4%, -0.0%] | 2 | +| Improvements ✅
(secondary) | -0.2% | [-0.4%, -0.1%] | 4 | +| All ❌✅ (primary) | -0.2% | [-0.4%, -0.0%] | 2 | + + +Rollup of 5 pull requests [#133086](https://github.com/rust-lang/rust/pull/133086) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=917a50a03931a9861c19a46f3e2a02a28f1da936&end=d3a4b1f46ba0fff6239e3d75abd285287ccd17f9&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | -4.2% | [-4.2%, -4.2%] | 1 | +| All ❌✅ (primary) | - | - | 0 | + + +Rollup of 5 pull requests [#133179](https://github.com/rust-lang/rust/pull/133179) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=c602e9aeaab636302d52b17da64e84044462a230&end=03ee4845197ce71aa5ee28cb937a3e863b18b42f&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.1% | [-0.2%, -0.1%] | 3 | +| Improvements ✅
(secondary) | -0.3% | [-0.4%, -0.2%] | 2 | +| All ❌✅ (primary) | -0.1% | [-0.2%, -0.1%] | 3 | + + +#### Mixed + +Rollup of 7 pull requests [#132919](https://github.com/rust-lang/rust/pull/132919) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=81eef2d362a6f03db6f8928f82d94298d31eb81b&end=67f21277cd40cd12e69aa34089f4a20926fd6dc5&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.4% | [0.1%, 1.3%] | 6 | +| Regressions ❌
(secondary) | 0.2% | [0.1%, 0.2%] | 2 | +| Improvements ✅
(primary) | -0.1% | [-0.1%, -0.1%] | 1 | +| Improvements ✅
(secondary) | -2.4% | [-2.4%, -2.4%] | 1 | +| All ❌✅ (primary) | 0.3% | [-0.1%, 1.3%] | 7 | + +* primary regressions are to doc scenarios; further investigation has indicated that PR #132144 is the root cause of them. +* pnkfelix's understanding is that this is known potential cost of progress when adding new traits (Receiver in this case) to stdlib. +* marked as triaged + +`#[inline]` integer parsing functions [#132870](https://github.com/rust-lang/rust/pull/132870) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=f7273e0044ad8f35ad27282e4ab776af50b61a54&end=b420d923cff05f51eb43f607f5d8dce827eeba97&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.9% | [0.1%, 5.1%] | 12 | +| Regressions ❌
(secondary) | 1.0% | [0.4%, 2.3%] | 9 | +| Improvements ✅
(primary) | -0.4% | [-0.6%, -0.2%] | 3 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | 0.6% | [-0.6%, 5.1%] | 15 | + +* main notable regressions were to regex opt-incr-patched (5%), cargo opt-full (1.14%), cranelift-codegen opt-full (0.83%) and ripgrep opt-full (0.74%); (everything else was <0.5% change). +* this is incurring a compilation cost, especially on opt builds, because its inlining more stuff +* pnkfelix assumes that the runtime benefit justifies this runtime cost. +* marking as triaged + +[perf] rustdoc: Perform less work when cleaning middle::ty parenthesized generic args [#132886](https://github.com/rust-lang/rust/pull/132886) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=44f233f2519ce5d633c87c38014d03d8a5f0e810&end=ec239b888f3945a6ce043db639f1741d14f08874&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 1.0% | [1.0%, 1.0%] | 2 | +| Regressions ❌
(secondary) | 0.2% | [0.2%, 0.2%] | 1 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | -0.2% | [-0.3%, -0.1%] | 5 | +| All ❌✅ (primary) | 1.0% | [1.0%, 1.0%] | 2 | + +* already identified as measurement bias/noise and marked as triaged; thanks lqd! + +Reland #132772: use `download-rustc="if-unchanged"` as a global default [#132872](https://github.com/rust-lang/rust/pull/132872) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=ec239b888f3945a6ce043db639f1741d14f08874&end=65b3877488ccfef3b1a903bfdd269559c59b957c&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | - | - | 0 | +| Regressions ❌
(secondary) | 0.2% | [0.2%, 0.3%] | 2 | +| Improvements ✅
(primary) | -1.1% | [-1.1%, -1.0%] | 3 | +| Improvements ✅
(secondary) | -0.5% | [-0.9%, -0.3%] | 6 | +| All ❌✅ (primary) | -1.1% | [-1.1%, -1.0%] | 3 | + +* already identified as measurement bias/noise and marked as triaged; thanks lqd! + +tweak attributes for const panic macro [#132662](https://github.com/rust-lang/rust/pull/132662) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=a00df61387e5389d6fe23e38e657f90d672668b1&end=8adb4b30f40e6fbd21dc1ba26c3301c7eeb6de3c&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.7% | [0.5%, 0.9%] | 2 | +| Regressions ❌
(secondary) | - | - | 0 | +| Improvements ✅
(primary) | -0.7% | [-2.0%, -0.2%] | 6 | +| Improvements ✅
(secondary) | - | - | 0 | +| All ❌✅ (primary) | -0.3% | [-2.0%, 0.9%] | 8 | + +* primary regressions are to image (0.92%) and webrender (0.50%) in opt-full scenarios. +* it seems like this might be just noise from changes to inlining decisions, since it seems like both those cases are undoing artificial improvements introduced by PR #132870 (noted above) +* marking as triaged + +rustc_metadata: Preprocess search paths for better performance [#132910](https://github.com/rust-lang/rust/pull/132910) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=f00f68245ebe85960000adaab18c1fe8da104cfe&end=76fd47124b8f8037b6187169b2cdf39139466952&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.0% | [0.0%, 0.0%] | 1 | +| Regressions ❌
(secondary) | 2.2% | [0.1%, 4.4%] | 2 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | -0.2% | [-0.2%, -0.2%] | 2 | +| All ❌✅ (primary) | 0.0% | [0.0%, 0.0%] | 1 | + +* already marked by saethlin as triaged: "The reported regressions are extremely marginal, and the discussion above: [#132910 (comment)](https://github.com/rust-lang/rust/pull/132910#issuecomment-2473240290) contains instructions for how to demonstrate a quite significant benefit of this change." + +Querify MonoItem collection [#132566](https://github.com/rust-lang/rust/pull/132566) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=5afd5ad29c014de69bea61d028a1ce832ed75a75&end=ee4a56e353dc3ddfcb12df5fe2dc1329a315c2f5&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:---------------:|:-----:| +| Regressions ❌
(primary) | 1.7% | [0.3%, 6.6%] | 10 | +| Regressions ❌
(secondary) | 1.7% | [0.3%, 3.7%] | 23 | +| Improvements ✅
(primary) | -6.3% | [-25.1%, -0.1%] | 73 | +| Improvements ✅
(secondary) | -9.9% | [-67.2%, -0.3%] | 22 | +| All ❌✅ (primary) | -5.4% | [-25.1%, 6.6%] | 83 | + +* massive improvements; many incremental benchmarks are improved by 5% to 20% (and even 25% in the case of ripgrep) +* the regressions here (mostly to helloworld) were anticipated during review and are far outweighed by the very notable gains in compiler performance across a broad swath of benchmarks. +* marking as triaged + +Likely unlikely fix [#120370](https://github.com/rust-lang/rust/pull/120370) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=5ec7d6eee7e0f5236ec1559499070eaf836bc608&end=3fb7e441aecc3c054d71eb4d752d06e7776e8888&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.9% | [0.1%, 2.8%] | 4 | +| Regressions ❌
(secondary) | 0.8% | [0.2%, 1.7%] | 18 | +| Improvements ✅
(primary) | -0.4% | [-1.9%, -0.1%] | 21 | +| Improvements ✅
(secondary) | -0.3% | [-0.4%, -0.1%] | 11 | +| All ❌✅ (primary) | -0.2% | [-1.9%, 2.8%] | 25 | + +* primary regressions on html5ever opt-full (2.78%), image opt-incr-patched (0.35%) (and stm32f4 doc-full (0.11%) but that's probably noise) +* already triaged by saethlin; "some change is expected because these intrinsics used to do nothing" (i.e. they were broken before and are now believed to be fixed). Also clearly the benefits on the primary benchmarks is outweighing the limited primary regressions. + +Use `TypingMode` throughout the compiler instead of `ParamEnv` [#132460](https://github.com/rust-lang/rust/pull/132460) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=03ee4845197ce71aa5ee28cb937a3e863b18b42f&end=b71fb5edc0217eaf8fc824a44cd7b0945e29ff4d&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 0.2% | [0.1%, 0.4%] | 8 | +| Regressions ❌
(secondary) | 0.9% | [0.2%, 2.7%] | 6 | +| Improvements ✅
(primary) | -0.3% | [-0.3%, -0.3%] | 1 | +| Improvements ✅
(secondary) | -0.8% | [-0.9%, -0.7%] | 5 | +| All ❌✅ (primary) | 0.1% | [-0.3%, 0.4%] | 9 | + +* primary regressions to unicode-normalization profiles and (to a lesser extent) diesel, stm32f4, and cargo. +* it seems like this regression was to some extent anticipated, at least from the [perf run that was done](https://github.com/rust-lang/rust/pull/132460#issuecomment-2465546736) while the PR was in development. +* not formally marking as triaged yet; instead, left note saying that I'm inclined to mark as triage if either the PR author or the PR reviewer agree that this "regression", such as it is, was anticipated. + +Improve VecCache under parallel frontend [#124780](https://github.com/rust-lang/rust/pull/124780) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=b71fb5edc0217eaf8fc824a44cd7b0945e29ff4d&end=5926e82dd1eae211c6e2ffe446de54df04798e89&stat=instructions:u) + +| (instructions:u) | mean | range | count | +|:----------------------------------:|:-----:|:--------------:|:-----:| +| Regressions ❌
(primary) | 1.0% | [0.2%, 3.3%] | 214 | +| Regressions ❌
(secondary) | 1.0% | [0.2%, 2.9%] | 172 | +| Improvements ✅
(primary) | - | - | 0 | +| Improvements ✅
(secondary) | -1.9% | [-3.0%, -0.4%] | 8 | +| All ❌✅ (primary) | 1.0% | [0.2%, 3.3%] | 214 | + +* left a note pointing out that there was a pretty serious process deviation, in that this was marked with rustc-perf-triaged for a different commit (maybe a different commit series?), months before this had gotten review or even landed... +* but having said that, I looked over the data and it seems like the label remains justified, so I have left the label in place. + +#### Untriaged Pull Requests + +- [#133120 Rollup of 7 pull requests](https://github.com/rust-lang/rust/pull/133120) +- [#132919 Rollup of 7 pull requests](https://github.com/rust-lang/rust/pull/132919) +- [#132870 `#[inline]` integer parsing functions](https://github.com/rust-lang/rust/pull/132870) +- [#132831 Rollup of 6 pull requests](https://github.com/rust-lang/rust/pull/132831) +- [#132662 tweak attributes for const panic macro](https://github.com/rust-lang/rust/pull/132662) +- [#132566 Querify MonoItem collection](https://github.com/rust-lang/rust/pull/132566) +- [#132542 add const_panic macro to make it easier to fall back to non-formatting panic in const](https://github.com/rust-lang/rust/pull/132542) +- [#132460 Use `TypingMode` throughout the compiler instead of `ParamEnv`](https://github.com/rust-lang/rust/pull/132460) +- [#132326 Rollup of 6 pull requests](https://github.com/rust-lang/rust/pull/132326) +- [#132116 Rollup of 4 pull requests](https://github.com/rust-lang/rust/pull/132116) +- [#131984 Stabilize if_let_rescope](https://github.com/rust-lang/rust/pull/131984) +- [#131721 Add new unstable feature `const_eq_ignore_ascii_case`](https://github.com/rust-lang/rust/pull/131721) +- [#131263 Introduce SolverRelating type relation to the new solver](https://github.com/rust-lang/rust/pull/131263) +- [#131111 Rollup of 4 pull requests](https://github.com/rust-lang/rust/pull/131111) +- [#130091 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/130091) +- [#129831 Rollup of 11 pull requests](https://github.com/rust-lang/rust/pull/129831) +- [#129817 Rollup of 12 pull requests](https://github.com/rust-lang/rust/pull/129817) +- [#129797 Try to reduce space usage in dist CI](https://github.com/rust-lang/rust/pull/129797) +- [#129790 Bump to 1.83](https://github.com/rust-lang/rust/pull/129790) +- [#129632 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/129632) +- [#129063 Apply size optimizations to panic machinery and some cold functions](https://github.com/rust-lang/rust/pull/129063) +- [#128936 Support reading thin archives in ArArchiveBuilder](https://github.com/rust-lang/rust/pull/128936) +- [#128443 Properly mark loop as diverging if it has no breaks](https://github.com/rust-lang/rust/pull/128443) +- [#128357 Detect non-lifetime binder params shadowing item params](https://github.com/rust-lang/rust/pull/128357) +- [#128301 Rollup of 3 pull requests](https://github.com/rust-lang/rust/pull/128301) +- [#128299 Simplify the canonical clone method and the copy-like forms to copy](https://github.com/rust-lang/rust/pull/128299) +- [#128169 Rollup of 5 pull requests](https://github.com/rust-lang/rust/pull/128169) +- [#128155 Rollup of 8 pull requests](https://github.com/rust-lang/rust/pull/128155) +- [#128104 Not lint pub structs without pub constructors intentionally](https://github.com/rust-lang/rust/pull/128104) +- [#127998 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/127998)