FindBLAS + MKL + Ctest fails on MacOS (SIP)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

FindBLAS + MKL + Ctest fails on MacOS (SIP)

Luke D'Alessandro
Hi all,

I have one of those complex interactions that results in an issue where it’s not clear to me which component is responsible.

I have unit tests written using CTest that depend on Intel's MKL BLAS implementation. I use find_package(BLAS) and link the test executables to ${BLAS_LIBRARIES}. The test executables depend on the DYLD_LIBRARY_PATH to find the mkl libraries, rather than an embedded LC_RPATH.

Unfortunately, because of Apple’s SIP, the DYLD_LIBRARY_PATH is not propagated to the ctest environment, so when it tries to run the tests it fails to link the MKL libraries.

I need to get CMake to embed an external LC_PATH for test executables in the build directory.

Here's a basic test executable (test.cpp)

#include <mkl_cblas.h>

int main() {
  return &cblas_dgemm != nullptr;
}
and here is the associated CMakeLists.txt

cmake_minimum_required(VERSION 3.14.5)

project(blas LANGUAGES CXX)
include(CTest)

set(BLA_VENDOR Intel10_64lp_seq)
find_package(BLAS REQUIRED)

add_executable(test_blas test.cpp)
target_link_libraries(test_blas ${BLAS_LIBRARIES})

add_test(NAME test_direct COMMAND test_blas)
This finds MKL and builds and links the executable without any problem, and I can run it manually from my shell. But when I run with CTest I have the SIP issue because MacOS won’t propagate the necessary DYLD_LIBRARY_PATH into CTest’s environment.

Lukes-MacBook:test ldalessa$ make test
Running tests...
Test project /Users/ldalessa/test
    Start 1: test_direct
1/1 Test #1: test_direct ......................Child aborted***Exception:   0.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.01 sec

The following tests FAILED:
      1 - test_direct (Child aborted)
Errors while running CTest
make: *** [test] Error 8

Lukes-MacBook:test ldalessa$ cat Testing/Temporary/LastTest.log 
Start testing: Jun 10 03:35 PDT
----------------------------------------------------------
1/1 Testing: test_direct
1/1 Test: test_direct
Command: "/Users/ldalessa/test/test_blas"
Directory: /Users/ldalessa/test
"test_direct" start time: Jun 10 03:35 PDT
Output:
----------------------------------------------------------
dyld: Library not loaded: @rpath/libmkl_intel_lp64.dylib
  Referenced from: /Users/ldalessa/test/test_blas
  Reason: image not found
<end of output>
Test time =   0.01 sec
----------------------------------------------------------
Test Failed.
"test_direct" end time: Jun 10 03:35 PDT
"test_direct" time elapsed: 00:00:00
----------------------------------------------------------

End testing: Jun 10 03:35 PDT
Lukes-MacBook:test ldalessa$ 
My temporary workaround for this is currently to use Apple's Accelerate framework instead of MKL, which links using an embedded LC_PATH properly. The reason this isn’t adequate is that it is an intrusive solution. Intel's MKL uses different headers (mkl_blas.hmkl_lapack.h) and symbol names which are explicitly used in the source code, so I have to have extra configure-time and configured header changes to adapt the code base to Accelerate. We also don’t (and won’t) have verified and validated code with Accelerate so it’s not a long term solution to our issues. Disabling SIP is also a non-starter.

Thanks,
Luke



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindBLAS + MKL + Ctest fails on MacOS (SIP)

Eric Dönges

The simplest way to do this is to use the BUILD_RPATH and/or INSTALL_RPATH properties, i.e. something like:

list(GET BLAS_LIBRARIES 0 _BLAS_FIRSTLIB)
get_filename_component(_BLAS_LIBDIR "${_BLAS_FIRSTLIB}" DIRECTORY
set_target_properties(test_blas PROPERTIES BUILD_RPATH "${_BLAS_LIBDIR}")

This assumes BLAS_LIBRARIES is a list of libraries specified with absolute paths; if this assumption is incorrect, you must figure out the directory to specify to BUILD_RPATH some other way. If the tests are run using the installed binary (or you build your binaries using the install rpath from the start, like we do), then you must set INSTALL_RPATH instead of BUILD_RPATH. For a detailed description, see the CMake wiki page on RPATH handling.

Also note that while I think this should work, I haven't actually tested it myself.

Am 09.07.19 um 16:22 schrieb Luke D'Alessandro:
Hi all,

I have one of those complex interactions that results in an issue where it’s not clear to me which component is responsible.

I have unit tests written using CTest that depend on Intel's MKL BLAS implementation. I use find_package(BLAS) and link the test executables to ${BLAS_LIBRARIES}. The test executables depend on the DYLD_LIBRARY_PATH to find the mkl libraries, rather than an embedded LC_RPATH.

Unfortunately, because of Apple’s SIP, the DYLD_LIBRARY_PATH is not propagated to the ctest environment, so when it tries to run the tests it fails to link the MKL libraries.

I need to get CMake to embed an external LC_PATH for test executables in the build directory.

Here's a basic test executable (test.cpp)

#include <mkl_cblas.h>

int main() {
  return &cblas_dgemm != nullptr;
}
and here is the associated CMakeLists.txt

cmake_minimum_required(VERSION 3.14.5)

project(blas LANGUAGES CXX)
include(CTest)

set(BLA_VENDOR Intel10_64lp_seq)
find_package(BLAS REQUIRED)

add_executable(test_blas test.cpp)
target_link_libraries(test_blas ${BLAS_LIBRARIES})

add_test(NAME test_direct COMMAND test_blas)
This finds MKL and builds and links the executable without any problem, and I can run it manually from my shell. But when I run with CTest I have the SIP issue because MacOS won’t propagate the necessary DYLD_LIBRARY_PATH into CTest’s environment.

Lukes-MacBook:test ldalessa$ make test
Running tests...
Test project /Users/ldalessa/test
    Start 1: test_direct
1/1 Test #1: test_direct ......................Child aborted***Exception:   0.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.01 sec

The following tests FAILED:
      1 - test_direct (Child aborted)
Errors while running CTest
make: *** [test] Error 8

Lukes-MacBook:test ldalessa$ cat Testing/Temporary/LastTest.log 
Start testing: Jun 10 03:35 PDT
----------------------------------------------------------
1/1 Testing: test_direct
1/1 Test: test_direct
Command: "/Users/ldalessa/test/test_blas"
Directory: /Users/ldalessa/test
"test_direct" start time: Jun 10 03:35 PDT
Output:
----------------------------------------------------------
dyld: Library not loaded: @rpath/libmkl_intel_lp64.dylib
  Referenced from: /Users/ldalessa/test/test_blas
  Reason: image not found
<end of output>
Test time =   0.01 sec
----------------------------------------------------------
Test Failed.
"test_direct" end time: Jun 10 03:35 PDT
"test_direct" time elapsed: 00:00:00
----------------------------------------------------------

End testing: Jun 10 03:35 PDT
Lukes-MacBook:test ldalessa$ 
My temporary workaround for this is currently to use Apple's Accelerate framework instead of MKL, which links using an embedded LC_PATH properly. The reason this isn’t adequate is that it is an intrusive solution. Intel's MKL uses different headers (mkl_blas.hmkl_lapack.h) and symbol names which are explicitly used in the source code, so I have to have extra configure-time and configured header changes to adapt the code base to Accelerate. We also don’t (and won’t) have verified and validated code with Accelerate so it’s not a long term solution to our issues. Disabling SIP is also a non-starter.

Thanks,
Luke



--

Dr. Eric Dönges
Senior Software Engineer

MVTec Software GmbH | Arnulfstr. 205 | 80634 Munich | Germany
[hidden email] | Tel: +49 89 457 695-0 | www.mvtec.com

 Sign up for our MVTec Newsletter!

Geschäftsführer: Dr. Wolfgang Eckstein, Dr. Olaf Munkelt
Amtsgericht München HRB 114695

 

MVTec Software GmbH Logo

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindBLAS + MKL + Ctest fails on MacOS (SIP)

Luke D'Alessandro


On Jul 10, 2019, at 4:42 AM, Eric Doenges <[hidden email]> wrote:

The simplest way to do this is to use the BUILD_RPATH and/or INSTALL_RPATH properties, i.e. something like:

list(GET BLAS_LIBRARIES 0 _BLAS_FIRSTLIB)
get_filename_component(_BLAS_LIBDIR "${_BLAS_FIRSTLIB}" DIRECTORY
set_target_properties(test_blas PROPERTIES BUILD_RPATH "${_BLAS_LIBDIR}")

This assumes BLAS_LIBRARIES is a list of libraries specified with absolute paths; if this assumption is incorrect, you must figure out the directory to specify to BUILD_RPATH some other way. If the tests are run using the installed binary (or you build your binaries using the install rpath from the start, like we do), then you must set INSTALL_RPATH instead of BUILD_RPATH. For a detailed description, see the CMake wiki page on RPATH handling.

Also note that while I think this should work, I haven't actually tested it myself.


Thank you Eric, this does work. I have a large number of test executables and more are always being added by users, so I appended the result to the CMAKE_BUILD_RPATH in the test root directory and this results in the expected behavior.

Luke

Am 09.07.19 um 16:22 schrieb Luke D'Alessandro:
Hi all,

I have one of those complex interactions that results in an issue where it’s not clear to me which component is responsible.

I have unit tests written using CTest that depend on Intel's MKL BLAS implementation. I use find_package(BLAS) and link the test executables to ${BLAS_LIBRARIES}. The test executables depend on the DYLD_LIBRARY_PATH to find the mkl libraries, rather than an embedded LC_RPATH.

Unfortunately, because of Apple’s SIP, the DYLD_LIBRARY_PATH is not propagated to the ctest environment, so when it tries to run the tests it fails to link the MKL libraries.

I need to get CMake to embed an external LC_PATH for test executables in the build directory.

Here's a basic test executable (test.cpp)

#include <mkl_cblas.h>

int main() {
  return &cblas_dgemm != nullptr;
}
and here is the associated CMakeLists.txt

cmake_minimum_required(VERSION 3.14.5)

project(blas LANGUAGES CXX)
include(CTest)

set(BLA_VENDOR Intel10_64lp_seq)
find_package(BLAS REQUIRED)

add_executable(test_blas test.cpp)
target_link_libraries(test_blas ${BLAS_LIBRARIES})

add_test(NAME test_direct COMMAND test_blas)
This finds MKL and builds and links the executable without any problem, and I can run it manually from my shell. But when I run with CTest I have the SIP issue because MacOS won’t propagate the necessary DYLD_LIBRARY_PATH into CTest’s environment.

Lukes-MacBook:test ldalessa$ make test
Running tests...
Test project /Users/ldalessa/test
    Start 1: test_direct
1/1 Test #1: test_direct ......................Child aborted***Exception:   0.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.01 sec

The following tests FAILED:
      1 - test_direct (Child aborted)
Errors while running CTest
make: *** [test] Error 8

Lukes-MacBook:test ldalessa$ cat Testing/Temporary/LastTest.log 
Start testing: Jun 10 03:35 PDT
----------------------------------------------------------
1/1 Testing: test_direct
1/1 Test: test_direct
Command: "/Users/ldalessa/test/test_blas"
Directory: /Users/ldalessa/test
"test_direct" start time: Jun 10 03:35 PDT
Output:
----------------------------------------------------------
dyld: Library not loaded: @rpath/libmkl_intel_lp64.dylib
  Referenced from: /Users/ldalessa/test/test_blas
  Reason: image not found
<end of output>
Test time =   0.01 sec
----------------------------------------------------------
Test Failed.
"test_direct" end time: Jun 10 03:35 PDT
"test_direct" time elapsed: 00:00:00
----------------------------------------------------------

End testing: Jun 10 03:35 PDT
Lukes-MacBook:test ldalessa$ 
My temporary workaround for this is currently to use Apple's Accelerate framework instead of MKL, which links using an embedded LC_PATH properly. The reason this isn’t adequate is that it is an intrusive solution. Intel's MKL uses different headers (mkl_blas.hmkl_lapack.h) and symbol names which are explicitly used in the source code, so I have to have extra configure-time and configured header changes to adapt the code base to Accelerate. We also don’t (and won’t) have verified and validated code with Accelerate so it’s not a long term solution to our issues. Disabling SIP is also a non-starter.

Thanks,
Luke



--

Dr. Eric Dönges
Senior Software Engineer

MVTec Software GmbH | Arnulfstr. 205 | 80634 Munich | Germany
[hidden email] | Tel: +49 89 457 695-0 | www.mvtec.com

 Sign up for our MVTec Newsletter!

Geschäftsführer: Dr. Wolfgang Eckstein, Dr. Olaf Munkelt
Amtsgericht München HRB 114695

 

MVTec Software GmbH Logo


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake