-
Notifications
You must be signed in to change notification settings - Fork 276
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix] #3928: Make failing the wasm tests actually fail the CI #3933
[fix] #3928: Make failing the wasm tests actually fail the CI #3933
Conversation
Pull Request Test Coverage Report for Build 6416151955
💛 - Coveralls |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see you also opened a PR to webassembly-test-runner
repo. Do you think we should use it when your fix is merged?
Yeah, I think it's fine to use it when/if it'll get merged. But, AFAIK, matklad is not doing any rust stuff recently, so it might be a while. |
295565f
to
2ae9df3
Compare
The |
@Arjentix can you help Nikita debug this? |
Will try |
I have some progress, but not much: it's probably a result of some unsoundness, or a compiler bug, as modifying the code (like inserting a call to a log function) can make the failure disappear. It also seems to occur in release regardless of whether the incremental compilation is used |
It's a double free caused by the confusion abt whether the function should take ownership. While debugging this, I've added ad-hoc logging capabilities to the test runner. Not sure if this logger should stay. (It would allow us to move back to the upstream |
d0fa213
to
4b9d8ac
Compare
Removed the ad-hoc logging system & made another PR to the upstream (for printing the backtrace of test failures). If it gets merged I'll get rid of the |
ok, since this PR is unlikely to conflict with any of the current work we can wait a few days before merging |
4b9d8ac
to
0a66b3e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks for a detailed explanation, hopefully we can rebuild the image soon :-)
0a66b3e
to
8061105
Compare
Reverted to hosting the wasm runner tool in monorepo, as the upstream would still be effectively only maintained by us |
8061105
to
f61aaf0
Compare
…il the CI Signed-off-by: Nikita Strygin <[email protected]>
The `log` and `dbg` functions do not take the pointer ownership, but their mock versions used for testing did Signed-off-by: Nikita Strygin <[email protected]>
f61aaf0
to
47b2998
Compare
Description
Add a new
iroha_wasm_test_runner
tool. It mostly duplicates thewebassembly-test-runner
, but actually returns a non-zero exit code if there are any test failures.This tool is expected to be installed on the developer's system with
cargo install
. It's not going to change often, so I don't think there's a risk of it going out-of-sync with the state of the repo.Also fix the
evaluate_expression
test, which was never passing before due to a typo.Linked issue
Closes #3928
Benefits
Alternatives
An alternative would be to continue to use the upstream test runner when/if matklad/webassembly-test#1 gets merged, but I wouldn't bet on it.
Checklist