TPLs BLAS and LAPACK: Add OpenBLAS to library names #13567
+5
−33
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cgcgcg
added
the
PA: Framework
bartlettroscoe
requested changes
Nov 1, 2024
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request. |
This allows to use OpenBLAS for BLAS and LAPACK without additional configuration. Signed-off-by: Christian Glusa <[email protected]>
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: cgcgcg |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
|
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: cgcgcg |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
Unfortunately, it looks like you may not be able to just rip out the specification of BLAS and LAPACK and let configure find and in the configure output here showing: But it seems harmless to try to rip all of this out and see what happens on all of these platforms. |
In a desire to make Trilinos easier to configure by default, try to use the default finds for BLAS and LAPACK. Signed-off-by: Roscoe A. Bartlett <[email protected]>
I am giving that a try with c597034. Let's see how that goes :-) |
bartlettroscoe
approved these changes
Nov 2, 2024
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: cgcgcg |
|
NOTICE: The AutoTester has encountered an internal error (usually a Communications Timeout), testing will be restarted, previous tests may still be running but will be ignored by the AutoTester... |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
Using Repos:
Pull Request Author: cgcgcg |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: PR_gcc-openmpi-openmp
Jenkins Parameters
Build InformationTest Name: PR_gcc
Jenkins Parameters
Build InformationTest Name: PR_gcc-openmpi_debug
Jenkins Parameters
Build InformationTest Name: PR_clang
Jenkins Parameters
Build InformationTest Name: Trilinos_PR_python3
Jenkins Parameters
Build InformationTest Name: PR_cuda
Jenkins Parameters
Build InformationTest Name: PR_intel
Jenkins Parameters
Build InformationTest Name: PR_cuda-uvm
Jenkins Parameters
|
|
Wow, only the CUDA build showed any errors after ripping out the BLAS and LAPACK specs: showing build errors like [here[(https://trilinos-cdash.sandia.gov/viewBuildError.php?buildid=1936670) showing: So when you are using a static BLAS, you need to be pulling in But the other builds are now showing default finds for BLAS and LAPACK like for the build: showing: I don't see any examples of where openblas was find. I guess that is just in the container and we don't see any container builds in current PR testing? |
|
If we really want to improve the default BLAS and LAPACK, find, we should just pull the trigger on switching to That will change the default find but will maintain behavior for customers that are explicitly specifying the libraries. |
|
I started prototyping that here: sebrowne@d1e0168 but have not tested it exhaustively yet. |
|
Yes, we only use openblas in the containers. The static library dependencies are the only risk here, as identified by the PR testing. If we want to go this route, we could leave those configuration settings in-place for the CUDA builds I suppose. |
@sebrowne, where can we see the containers configuration and build output on GHA? I looked and I can't seem to find them. |
Here is our configuration repository. Those are planned to be moved into the main repository here (hopefully very soon). Action output is here (for example, I chose one). If you look here you can see the summary, which includes the container image used |
|
The tricky thing is that this also doesn't play super nicely with the TriBITS behavior engaged with |
@sebrowne, how so? Can you give an example? |
sebrowne
reviewed
Nov 26, 2024
There was a problem hiding this comment.
After investigation, I think this is acceptable for now (eventually we should probably use the find_package() approach). I recommend leaving the specifications in-place for the AT1 specs, since there may be multiple BLAS/LAPACK on the system, whereas in our AT2 containers we can be more reasonably certain that it's using the specific one we want.
| @@ -1734,7 +1717,6 @@ opt-set-cmake-var Teko_DISABLE_LSCSTABALIZED_TPETRA_ALPAH_INV_D BOOL : O | |||
| opt-set-cmake-var CMAKE_CXX_FLAGS STRING : -fno-strict-aliasing -Wall -Wno-clobbered -Wno-vla -Wno-pragmas -Wno-unknown-pragmas -Wno-unused-local-typedefs -Wno-literal-suffix -Wno-deprecated-declarations -Wno-misleading-indentation -Wno-int-in-bool-context -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-inline -Wno-nonnull-compare -Wno-address | |||
|
|
|||
| # TPL_BLAS_LIBRARIES is redefined here with libm for SuperLU to properly link | |||
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING FORCE : -L${BLAS_ROOT|ENV}/lib;-lblas;-lgfortran;-lgomp;-lm | |||
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
| @@ -1608,13 +1593,11 @@ opt-set-cmake-var TPL_ENABLE_Zlib BOOL FORCE : ON | |||
|
|
|||
| #TPL_*_LIBRARIES | |||
| # see https://github.com/trilinos/Trilinos/issues/11109#issuecomment-1272146298 | |||
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING FORCE : /lib64/libblas.so.3 | |||
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
| opt-set-cmake-var TPL_BoostLib_LIBRARIES STRING FORCE : ${BOOST_LIB|ENV}/libboost_program_options.a;${BOOST_LIB|ENV}/libboost_system.a | ||
| opt-set-cmake-var TPL_Boost_LIBRARIES STRING FORCE : ${BOOST_LIB|ENV}/libboost_program_options.a;${BOOST_LIB|ENV}/libboost_system.a | ||
| opt-set-cmake-var TPL_DLlib_LIBRARIES FILEPATH FORCE : "-ldl" | ||
| opt-set-cmake-var TPL_HDF5_LIBRARIES STRING FORCE : ${HDF5_LIB|ENV}/libhdf5_hl.so;${HDF5_LIB|ENV}/libhdf5.a;${ZLIB_LIB|ENV}/libz.a;-ldl | ||
| # see https://github.com/trilinos/Trilinos/issues/11109#issuecomment-1272146298 | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARIES STRING FORCE : /lib64/liblapack.so.3 |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
| opt-set-cmake-var TPL_BoostLib_LIBRARIES STRING FORCE : ${BOOST_LIB|ENV}/libboost_program_options.a;${BOOST_LIB|ENV}/libboost_system.a | ||
| opt-set-cmake-var TPL_Boost_LIBRARIES STRING FORCE : ${BOOST_LIB|ENV}/libboost_program_options.a;${BOOST_LIB|ENV}/libboost_system.a | ||
| opt-set-cmake-var TPL_DLlib_LIBRARIES FILEPATH FORCE : "-ldl" | ||
| opt-set-cmake-var TPL_HDF5_LIBRARIES STRING FORCE : ${HDF5_LIB|ENV}/libhdf5_hl.so;${HDF5_LIB|ENV}/libhdf5.a;${ZLIB_LIB|ENV}/libz.a;-ldl | ||
| # see https://github.com/trilinos/Trilinos/issues/11109#issuecomment-1272146298 | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARIES STRING FORCE : /lib64/liblapack.so.3 |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
| @@ -1466,13 +1453,11 @@ opt-set-cmake-var TPL_ENABLE_Zlib BOOL FORCE : ON | |||
|
|
|||
| #TPL_*_LIBRARIES | |||
| # see https://github.com/trilinos/Trilinos/issues/11109#issuecomment-1272146298 | |||
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING FORCE : /lib64/libblas.so.3 | |||
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
Comment on lines
-1220
to
-1221
| opt-set-cmake-var TPL_LAPACK_LIBRARIES STRING FORCE : -L${BLAS_ROOT|ENV}/lib;-lopenblas;-lgfortran;-lgomp | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARY_DIRS STRING FORCE : ${BLAS_ROOT|ENV}/lib |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
Comment on lines
-1214
to
-1215
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING FORCE : -L${BLAS_ROOT|ENV}/lib;-lopenblas;-lgfortran;-lgomp | ||
| opt-set-cmake-var TPL_BLAS_LIBRARY_DIRS STRING FORCE : ${BLAS_ROOT|ENV}/lib |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
Comment on lines
-1178
to
-1181
| opt-set-cmake-var TPL_BLAS_LIBRARY_DIRS STRING FORCE : ${CBLAS_ROOT|ENV}/lib | ||
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING FORCE : ${CBLAS_ROOT|ENV}/lib/libblas.a;-L${CBLAS_ROOT|ENV}/lib;-lgfortran;-lgomp;-lm | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARY_DIRS STRING FORCE : ${LAPACK_ROOT|ENV}/lib | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARIES STRING FORCE : -L${LAPACK_ROOT|ENV}/lib;-lgfortran;-lgomp;${LAPACK_ROOT|ENV}/lib/liblapack.a |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
Comment on lines
-402
to
+400
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING FORCE : -L${BLAS_ROOT|ENV}/lib;-lblas;-lgfortran;-lgomp | ||
| opt-set-cmake-var TPL_BLAS_LIBRARY_DIRS STRING FORCE : ${BLAS_ROOT|ENV}/lib | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARIES STRING FORCE : -L${BLAS_ROOT|ENV}/lib;-llapack;-lgfortran;-lgomp | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARY_DIRS STRING FORCE : ${BLAS_ROOT|ENV}/lib | ||
| # Use default finds |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
Comment on lines
-395
to
-396
| opt-set-cmake-var TPL_BLAS_LIBRARIES STRING : ${BLAS_LIBRARIES|ENV} | ||
| opt-set-cmake-var TPL_LAPACK_LIBRARIES STRING : ${LAPACK_LIBRARIES|ENV} |
There was a problem hiding this comment.
Leave this (want to be specific on machines with multiple potential providers)
I went and read the documentation more carefully and tested it out. I did not know about the 'inner-space-separated quoted' behavior. So I take it back, this will probably work perfectly fine. |
Might be easier to specify what removals to keep 😊 (I just removed everything to see how many builds still worked after removing the explicit determinations of BLAS and LAPACK libraries.) |
|
@bartlettroscoe fair enough. I took a look through the file and found some other issues relating to inheritance (which has happened many times in the past), so I'm going to try and clean up all of them (if it's not too bad, I may just push it to this MR to help out, but if it's logically separable, I'll open a different PR). |
|
@sebrowne Feel free to push to it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@trilinos/framework @bartlettroscoe
Motivation
This allows to use OpenBLAS for BLAS and LAPACK without additional configuration.