Auto merge of #129230 - RalfJung:miri-sync, r=RalfJung

Miri subtree update

r? `@ghost`

src/tools/miri/.cargo/config.toml

@@ -0,0 +1,9 @@
+[unstable]
+profile-rustflags = true
+
+# Add back the containing directory of the packages we have to refer to using --manifest-path.
+# Per-package profiles avoid adding this to build dependencies.
+[profile.dev.package."cargo-miri"]
+rustflags = ["--remap-path-prefix", "=cargo-miri"]
+[profile.dev.package."miri-script"]
+rustflags = ["--remap-path-prefix", "=miri-script"]

src/tools/miri/.github/workflows/ci.yml

@@ -60,7 +60,7 @@ jobs:
       - name: clippy (all features)
         run: ./miri clippy --all-features -- -D warnings
       - name: rustdoc
-        run: RUSTDOCFLAGS="-Dwarnings" ./miri cargo doc --document-private-items
+        run: RUSTDOCFLAGS="-Dwarnings" ./miri doc --document-private-items
 
   # These jobs doesn't actually test anything, but they're only used to tell
   # bors the build completed, as there is no practical way to detect when a
@@ -123,6 +123,8 @@ jobs:
         run: |
           git config --global user.name 'The Miri Cronjob Bot'
           git config --global user.email '[email protected]'
+      - name: Install nightly toolchain
+        run: rustup toolchain install nightly --profile minimal
       - name: get changes from rustc
         run: ./miri rustc-pull
       - name: Install rustup-toolchain-install-master

src/tools/miri/.github/workflows/setup/action.yml

@@ -35,6 +35,10 @@ runs:
         run: cargo install -f rustup-toolchain-install-master hyperfine
         shell: bash
 
+      - name: Install nightly toolchain
+        run: rustup toolchain install nightly --profile minimal
+        shell: bash
+
       - name: Install "master" toolchain
         run: |
           if [[ ${{ github.event_name }} == 'schedule' ]]; then

src/tools/miri/CONTRIBUTING.md

@@ -8,10 +8,43 @@ find useful.
 Check out the issues on this GitHub repository for some ideas. In particular,
 look for the green `E-*` labels which mark issues that should be rather
 well-suited for onboarding. For more ideas or help with hacking on Miri, you can
-contact us (`oli-obk` and `RalfJ`) on the [Rust Zulip].
+contact us on the [Rust Zulip]. See the [Rust website](https://www.rust-lang.org/governance/teams/compiler#team-miri)
+for a list of Miri maintainers.
 
 [Rust Zulip]: https://rust-lang.zulipchat.com
 
+### Larger-scale contributions
+
+If you are thinking about making a larger-scale contribution -- in particular anything that needs
+more than can reasonably fit in a single PR to be feature-complete -- then please talk to us before
+writing significant amounts of code. Generally, we will ask that you follow a three-step "project"
+process for such contributions:
+
+1. Clearly define the **goal** of the project. This defines the scope of the project, i.e. which
+   part of which APIs should be supported. If this involves functions that expose a big API surface
+   with lots of flags, the project may want to support only a tiny subset of flags; that should be
+   documented. A good way to express the goal is with one or more test cases that Miri should be
+   able to successfully execute when the project is completed. It is a good idea to get feedback
+   from team members already at this stage to ensure that the project is reasonably scoped and
+   aligns with our interests.
+2. Make a **design** for how to realize the goal. A larger project will likely have to do global
+   changes to Miri, like adding new global state to the `Machine` type or new methods to the
+   `FileDescription` trait. Often we have to iterate on those changes, which can quite substantially
+   change how the final implementation looks like.
+
+    The design should be reasonably concrete, i.e. for new global state or methods the corresponding
+   Rust types and method signatures should be spelled out. We realize that it can be hard to make a
+   design without doing implementation work, in particular if you are not yet familiar with the
+   codebase. Doing draft implementations in phase 2 of this process is perfectly fine, just please
+   be aware that we might request fundamental changes that can require significantly reworking what
+   you already did. If you open a PR in this stage, please clearly indicate that this project is
+   still in the design stage.
+
+3. Finish the **implementation** and have it reviewed.
+
+This process is largely informal, and its primary goal is to more clearly communicate expectations.
+Please get in touch with us if you have any questions!
+
 ## Preparing the build environment
 
 Miri heavily relies on internal and unstable rustc interfaces to execute MIR,
@@ -173,24 +206,24 @@ to `.vscode/settings.json` in your local Miri clone:
         "cargo-miri/Cargo.toml",
         "miri-script/Cargo.toml",
     ],
+    "rust-analyzer.check.invocationLocation": "root",
+    "rust-analyzer.check.invocationStrategy": "once",
     "rust-analyzer.check.overrideCommand": [
         "env",
         "MIRI_AUTO_OPS=no",
         "./miri",
-        "cargo",
         "clippy", // make this `check` when working with a locally built rustc
         "--message-format=json",
-        "--all-targets",
     ],
     // Contrary to what the name suggests, this also affects proc macros.
+    "rust-analyzer.cargo.buildScripts.invocationLocation": "root",
+    "rust-analyzer.cargo.buildScripts.invocationStrategy": "once",
     "rust-analyzer.cargo.buildScripts.overrideCommand": [
         "env",
         "MIRI_AUTO_OPS=no",
         "./miri",
-        "cargo",
         "check",
         "--message-format=json",
-        "--all-targets",
     ],
 }
 ```
@@ -309,6 +342,7 @@ anyone but Miri itself. They are used to communicate between different Miri
 binaries, and as such worth documenting:
 
 * `CARGO_EXTRA_FLAGS` is understood by `./miri` and passed to all host cargo invocations.
+  It is reserved for CI usage; setting the wrong flags this way can easily confuse the script.
 * `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
   actually not interpret the code but compile it like rustc would. With `target`, Miri sets
   some compiler flags to prepare the code for interpretation; with `host`, this is not done.

src/tools/miri/README.md

@@ -414,10 +414,6 @@ to Miri failing to detect cases of undefined behavior in a program.
   being allocated or freed.  This helps in debugging memory leaks and
   use after free bugs. Specifying this argument multiple times does not overwrite the previous
   values, instead it appends its values to the list. Listing an id multiple times has no effect.
-* `-Zmiri-track-call-id=<id1>,<id2>,...` shows a backtrace when the given call ids are
-  assigned to a stack frame.  This helps in debugging UB related to Stacked
-  Borrows "protectors". Specifying this argument multiple times does not overwrite the previous
-  values, instead it appends its values to the list. Listing an id multiple times has no effect.
 * `-Zmiri-track-pointer-tag=<tag1>,<tag2>,...` shows a backtrace when a given pointer tag
   is created and when (if ever) it is popped from a borrow stack (which is where the tag becomes invalid
   and any future use of it will error).  This helps you in finding out why UB is

src/tools/miri/cargo-miri/src/util.rs

@@ -93,12 +93,9 @@ pub fn find_miri() -> PathBuf {
     if let Some(path) = env::var_os("MIRI") {
         return path.into();
     }
+    // Assume it is in the same directory as ourselves.
     let mut path = std::env::current_exe().expect("current executable path invalid");
-    if cfg!(windows) {
-        path.set_file_name("miri.exe");
-    } else {
-        path.set_file_name("miri");
-    }
+    path.set_file_name(format!("miri{}", env::consts::EXE_SUFFIX));
     path
 }
 

src/tools/miri/ci/ci.sh

@@ -150,8 +150,8 @@ case $HOST_TARGET in
     UNIX="panic/panic panic/unwind concurrency/simple atomic libc-mem libc-misc libc-random env num_cpus" # the things that are very similar across all Unixes, and hence easily supported there
     TEST_TARGET=x86_64-unknown-freebsd run_tests_minimal $BASIC $UNIX threadname libc-time fs
     TEST_TARGET=i686-unknown-freebsd   run_tests_minimal $BASIC $UNIX threadname libc-time fs
-    TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $BASIC $UNIX threadname pthread-sync available-parallelism libc-time
-    TEST_TARGET=x86_64-pc-solaris      run_tests_minimal $BASIC $UNIX threadname pthread-sync available-parallelism libc-time
+    TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $BASIC $UNIX threadname pthread-sync available-parallelism libc-time tls
+    TEST_TARGET=x86_64-pc-solaris      run_tests_minimal $BASIC $UNIX threadname pthread-sync available-parallelism libc-time tls
     TEST_TARGET=aarch64-linux-android  run_tests_minimal $BASIC $UNIX
     TEST_TARGET=wasm32-wasip2          run_tests_minimal empty_main wasm heap_alloc libc-mem
     TEST_TARGET=wasm32-unknown-unknown run_tests_minimal empty_main wasm

src/tools/miri/miri

@@ -1,8 +1,15 @@
 #!/usr/bin/env bash
 set -e
+# We want to call the binary directly, so we need to know where it ends up.
+MIRI_SCRIPT_TARGET_DIR="$(dirname "$0")"/miri-script/target
+# If stdout is not a terminal and we are not on CI, assume that we are being invoked by RA, and use JSON output.
+if ! [ -t 1 ] && [ -z "$CI" ]; then
+  MESSAGE_FORMAT="--message-format=json"
+fi
+# We need a nightly toolchain, for the `profile-rustflags` cargo feature.
+cargo +nightly build $CARGO_EXTRA_FLAGS --manifest-path "$(dirname "$0")"/miri-script/Cargo.toml \
+  -q --target-dir "$MIRI_SCRIPT_TARGET_DIR" $MESSAGE_FORMAT || \
+  ( echo "Failed to build miri-script. Is the 'nightly' toolchain installed?"; exit 1 )
 # Instead of doing just `cargo run --manifest-path .. $@`, we invoke miri-script binary directly. Invoking `cargo run` goes through
 # rustup (that sets it's own environmental variables), which is undesirable.
-MIRI_SCRIPT_TARGET_DIR="$(dirname "$0")"/miri-script/target
-cargo +stable build $CARGO_EXTRA_FLAGS -q --target-dir "$MIRI_SCRIPT_TARGET_DIR" --manifest-path "$(dirname "$0")"/miri-script/Cargo.toml || \
-  ( echo "Failed to build miri-script. Is the 'stable' toolchain installed?"; exit 1 )
 "$MIRI_SCRIPT_TARGET_DIR"/debug/miri-script "$@"