Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix "./onnxruntime_test_all --help" segfault #22839

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Fix "./onnxruntime_test_all --help" segfault #22839

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7a50361
Fix "./onnxruntime_test_all --help" segfault
amancini-N Aug 23, 2024
85437a0
Restore setup order and always do teardown at exit
amancini-N Nov 18, 2024
7dd8ff6
avoid atexit, skip setup if --help
amancini-N Nov 25, 2024
ea162ff
undo space change
amancini-N Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions onnxruntime/test/unittest_main/test_main.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ int TEST_MAIN(int argc, char** argv) {

ORT_TRY {
ortenv_setup();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very tricky. The ::testing::InitGoogleTest function also calls a lot of initialization functions in the tests, and the these functions may call into ONNX Runtime. Therefore ortenv should be initialized before that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I pushed an alternative restoring the original order, but making sure ortenv teardown is always done. This one also seems working with "--help" case

// TODO: Fix the C API issue
atexit(ortenv_teardown); // If we don't do this, it will crash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Atexit functions get executed after the main function is returned, especially , it is after the "::google::protobuf::ShutdownProtobufLibrary" , which means in the teardown function we cannot call all protobuf functions, which is not possible. Also, it will also change the destruction order between function local statics and this "ortenv_teardown" function. Because basically the function local statics will use atexit function to register their destructors. Therefore, here the ordering is really very tricky. I would suggest keeping it unchanged unless there is an urgent need.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would using gsl::finally instead of atexit work?

e.g., #include <gsl/gsl>, then,

Suggested change
atexit(ortenv_teardown); // If we don't do this, it will crash
auto clean_up_ortenv = gsl::finally(ortenv_teardown); // If we don't do this, it will crash

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edgchen1 tried with gsl::finally, no luck. The problem is abseil parsing is just calling for exit() if --help flag is used, so only atexit has effect in this way.
@snnn I see your point, I agree this ordering is indeed tricky, and unfortunately abseil and gtest don't leave enough control on possible hooks on help exit.

I've pushed one last tentative. This time, ortenv_setup is called only if --help is not specified. This should not alter the ordering for test execution and --help is not crashing, as ortenv_setup is not called in that case

::testing::InitGoogleTest(&argc, argv);

status = RUN_ALL_TESTS();
Expand All @@ -104,9 +106,6 @@ int TEST_MAIN(int argc, char** argv) {
});
}

// TODO: Fix the C API issue
ortenv_teardown(); // If we don't do this, it will crash

#ifndef USE_ONNXRUNTIME_DLL
// make memory leak checker happy
::google::protobuf::ShutdownProtobufLibrary();
Expand Down
Loading