More work on zstd compression
#128935
Merged
More work on zstd compression
#128935
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustbot
added
A-run-make
|
My understanding is that test builders use The That builder should now have @bors try |
|
(Let me know what tests and try builds we'd want to double-check.) |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 10, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu
|
☀️ Try build successful - checks-actions |
rustbot
added
the
T-infra
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 10, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu
|
☀️ Try build successful - checks-actions |
|
Cool, the two tests (that now fail when a compression algorithm isn't found) still pass
|
Kobzol
reviewed
Aug 11, 2024
There was a problem hiding this comment.
This looks great! However, I'm not sure if we can ignore the LLVM build in test runners everytime. The runner uses llvm.download-ci-llvm=if-unchanged, so when there's a change to LLVM (or ), it will actually rebuild LLVM. In that case, we can either ignore the test (but it's not ideal, as the test would only run in the next merge), or make sure that we actually build LLVM with ZSTD support even in the test runner.rustc_llvm
Could you try to make a fake LLVM change to see what happens?
I pushed changes to Let's see what happens: @bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 11, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu
|
Looks like it downloaded LLVM anyway 😆 |
|
I sure hope switching to another repo won't download LLVM again! @bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 11, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu
|
Alright I pushed a different commit to the llvm fork, hopefully the builder will pick it now .... @bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 11, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu
|
☀️ Try build successful - checks-actions |
|
@bors r- The distcheck compressed-debuginfo test failed at #129547 (comment) |
bors
added
S-waiting-on-author
|
It seems distcheck doesn’t even have zlib enabled… |
also make it fail if there's a compression issue
it now checks zlib and zstd, via rustc and rust-lld
move it where it's used, and name it like the other scripts
but warn about it
|
I've removed the commit with the unconditional test change. We'll clean up zlib being randomly enabled later. @bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 25, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu try-job: x86_64-msvc try-job: x86_64-gnu-distcheck
|
☀️ Try build successful - checks-actions |
|
@bors r=Kobzol |
bors
added
S-waiting-on-bors
workingjubilee
added a commit
to workingjubilee/rustc
that referenced
this pull request
Aug 27, 2024
More work on `zstd` compression r? `@Kobzol` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu try-job: x86_64-msvc try-job: x86_64-gnu-distcheck
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 27, 2024
…kingjubilee Rollup of 9 pull requests Successful merges: - rust-lang#126985 (Implement `-Z embed-source` (DWARFv5 source code embedding extension)) - rust-lang#127922 (Add unsafe to extern blocks in style guide) - rust-lang#128731 (simd_shuffle intrinsic: allow argument to be passed as vector) - rust-lang#128935 (More work on `zstd` compression) - rust-lang#128942 (miri weak memory emulation: put previous value into initial store buffer) - rust-lang#129418 (rustc: Simplify getting sysroot library directory) - rust-lang#129490 (Add Trusty OS as tier 3 target) - rust-lang#129559 (float types: document NaN bit pattern guarantees) - rust-lang#129642 (Bump backtrace to rust-lang/backtrace@fc37b22) r? `@ghost` `@rustbot` modify labels: rollup
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 27, 2024
Rollup of 9 pull requests Successful merges: - rust-lang#126985 (Implement `-Z embed-source` (DWARFv5 source code embedding extension)) - rust-lang#127922 (Add unsafe to extern blocks in style guide) - rust-lang#128731 (simd_shuffle intrinsic: allow argument to be passed as vector) - rust-lang#128935 (More work on `zstd` compression) - rust-lang#128942 (miri weak memory emulation: put previous value into initial store buffer) - rust-lang#129418 (rustc: Simplify getting sysroot library directory) - rust-lang#129490 (Add Trusty OS as tier 3 target) - rust-lang#129536 (Add `f16` and `f128` inline ASM support for `aarch64`) - rust-lang#129559 (float types: document NaN bit pattern guarantees) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Aug 27, 2024
Rollup merge of rust-lang#128935 - lqd:needs-zstd, r=Kobzol More work on `zstd` compression r? ``@Kobzol`` as we've discussed this. This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional. Knowing whether llvm/lld was built with `LLVM_ENABLE_ZSTD` is quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case. zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported. The zstd test is made unconditional in what it tests, by relying on `needs-llvm-zstd` to be ignored when `llvm.libzstd` isn't enabled in `config.toml`. try-job: x86_64-gnu try-job: x86_64-msvc try-job: x86_64-gnu-distcheck
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
r? @Kobzol as we've discussed this.
This is a draft to show the current approach of supporting zstd in compiletest, and making the tests using it unconditional.
Knowing whether llvm/lld was built with
LLVM_ENABLE_ZSTDis quite hard, so there are two strategies. There are details in the code, and we can discuss this approach. Until we know the config used to build CI artifacts, it seems our options are somewhat limited in any case.zlib compression seems always enabled, so we only check this in its dedicated test, allowing the test to ignore errors due to zstd not being supported.
The zstd test is made unconditional in what it tests, by relying on
needs-llvm-zstdto be ignored whenllvm.libzstdisn't enabled inconfig.toml.try-job: x86_64-gnu
try-job: x86_64-msvc
try-job: x86_64-gnu-distcheck