fixup-bundle usability

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

fixup-bundle usability

Andreas Naumann
Dear CMakers,

recently I tried to bundle an application in Windows. From the
documentation [1] I see that I should provide the directories to the
non-system libraries.

But these information should be already in the properties of the
targets, arent they? Is there any extension in cmake, that provides
these paths?

How do other users handle dependencies to external dlls?


[1] https://cmake.org/cmake/help/v3.0/module/BundleUtilities.html

Regards,
Andreas

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: fixup-bundle usability

Francis Giraldeau
You are right, the fixup bundle is difficult to use. Here are some undocumented tips:

Put the install(CODE) with the fixup_bundle() call in a CMakeLists.txt in its own directory. In your main CMakeLists.txt file, add this directory with add_subdirectory() last. This install(CODE) will run after all the other install directive, otherwise the fixup_bundle() might run before other targets are installed.

The main thing is to build the library path variable. Yes, this information can be recovered from the target itself, but fixup_bundle() won't gather that info for you. Here is an example with Qt: 

get_target_property(QT_CORE_LIB Qt5::Core LOCATION)
get_filename_component(QT_RUNTIME_DIR "${QT_CORE_LIB}" DIRECTORY)
list(APPEND LIBS_PATH "${QT_RUNTIME_DIR}")

If you are using VTK, there is already a variable:
list(APPEND LIBS_PATH "${VTK_RUNTIME_LIBRARY_DIRS}")

You might as well run windeployqt (and similar tool for other platforms) inside install(CODE) script if you have a Qt app, such that the styles and the plugins and other stuff gets copied. 

execute_process(COMMAND windeployqt.exe --release \"\${MAIN_APP}\")

Workaround wrong tool detection using MinGW on Windows:
if(WIN32 AND NOT MSVC)
  set(GP_TOOL "objdump")
endif()

install(CODE "\
...
  include(BundleUtilities)
  set(gp_tool \"${GP_TOOL}\")
  fixup_bundle(\"\${MAIN_APP}\" \"\" \"${LIBS_PATH}\")
...

And there is the escaping... You have to escape quotes inside
the script. Escape the '$' sign if you want to refer to the
variable at install time. Remember that you don't have access
to configure-time variables at install time. Unescaped '$' prefix
will be replaced by its value at configure time, somewhat like
a macro or a template.

To see the generated script, look into ${CMAKE_BINARY_DIR} with
the directory name corresponding to the CMakeLists.txt containing
the install(CODE). Example:

${CMAKE_SOURCE_DIR}/cmake/bundle/CMakeLists.txt 
-> ${CMAKE_BINARY_DIR}/cmake/bundle/cmake_install.cmake

You can check the generated script, its very
handy when things go wrong.

Also, it is easier to put all libraries and binaries in their
own directory. I have this near the begining of my project's
CMakeLists.

set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)

If you have some library that you don't want in your bundle,
you might want to disable install for it, for instance,
google test:

add_subdirectory(3rdparty/googletest/ EXCLUDE_FROM_ALL)

I'm using "ntldd.exe -R" on Windows to check the dependencies manually
(its very much like ldd on Linux). It is much more handy than
dependency walker.

Fixup_bundle is not magical either. The library dependencies must be
known beforehand. It means that if you do some dlopen() tricks at
runtime, fixup_bundle() cannot know that and you will have to install
these libraries manually yourself. One notable example of this is
Intel MKL using the Single Dynamic Library (mkl_rt). Using this
library entry point simplifies the linking and will load the
best library at runtime depending on the hardware. Fixup_bundle() will
copy only mkl_rt and you have to copy the other mkl libraries to
avoid runtime error.

Fixup bundle is tricky to put in place, but having a fixed list
of libraries to copy is even more cumbersome and flaky. When
supplied with the proper directories, the bundle is right every time.

Francis

Le sam. 16 févr. 2019 à 04:04, Andreas Naumann <[hidden email]> a écrit :
Dear CMakers,

recently I tried to bundle an application in Windows. From the
documentation [1] I see that I should provide the directories to the
non-system libraries.

But these information should be already in the properties of the
targets, arent they? Is there any extension in cmake, that provides
these paths?

How do other users handle dependencies to external dlls?


[1] https://cmake.org/cmake/help/v3.0/module/BundleUtilities.html

Regards,
Andreas

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Francis Giraldeau

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: fixup-bundle usability

Andreas Naumann
Thank you very much for your explaination. At the moment, I link only to
boost and some in-house libraries. When experimenting with CMake and
reading the docs, I got the same impression as you described. And I
hoped to miss something obvious.

When I understand you correctly, one have to set the directories from
hand. Furthermore there are additional variables (in case of VTK), which
contain the library directories. But modern CMake works with targets and
properties and let the dependencies propagate.

To test the idea a little bit, I startet with the following function

function(setBundle targetName dest)
   install(TARGETS ${targetName} DESTINATION ${dest})

   get_target_property(targtDeps ${targetName} LINK_LIBRARIES)
   file(TO_CMAKE_PATH "${CMAKE_INSTALL_PREFIX}" instPref)
   set(depDirs )
   foreach(t IN LISTS targtDeps)
     #disable the INTERFACE target due to error later.
     if(TARGET ${t} AND NOT ("${t}" STREQUAL "Boost::disable_autolinking"))
       get_target_property(isImported ${t} IMPORTED)
       if(isImported)
         get_target_property(depFile ${t} IMPORTED_LOCATION_DEBUG)
         if(depFile)
           get_filename_component(cdepDir ${depFile} DIRECTORY)
           list(APPEND depDirs ${cdepDir})
         endif()
       endif()
     endif()
   endforeach()
   install(CODE "include(BundleUtilities)
fixup_bundle(\"${instPref}/${dest}/$<TARGET_FILE_NAME:${targetName}>\"
\"\" \"${depDirs}\")")
endfunction()

It is far away from optimal and has some drawbacks, especially the
detection of interface libraries and the location of the imported target
should be more fail proof.

How can I ask for feature requests on gitlab? I found the issue tracker,
but nothing in regard for a feature or improvement.

Andreas

Am 18.02.19 um 15:56 schrieb Francis Giraldeau:

> You are right, the fixup bundle is difficult to use. Here are some
> undocumented tips:
>
> Put the install(CODE) with the fixup_bundle() call in a CMakeLists.txt
> in its own directory. In your main CMakeLists.txt file, add this
> directory with add_subdirectory() last. This install(CODE) will run
> after all the other install directive, otherwise the fixup_bundle()
> might run before other targets are installed.
>
> The main thing is to build the library path variable. Yes, this
> information can be recovered from the target itself, but
> fixup_bundle() won't gather that info for you. Here is an example with
> Qt:
>
> get_target_property(QT_CORE_LIBQt5::CoreLOCATION)
> get_filename_component(QT_RUNTIME_DIR"${QT_CORE_LIB}"DIRECTORY)
> list(APPENDLIBS_PATH"${QT_RUNTIME_DIR}")
>
> If you are using VTK, there is already a variable:
> list(APPENDLIBS_PATH"${VTK_RUNTIME_LIBRARY_DIRS}")
>
> You might as well run windeployqt (and similar tool for other
> platforms) inside install(CODE) script if you have a Qt app, such that
> the styles and the plugins and other stuff gets copied.
>
> execute_process(COMMANDwindeployqt.exe--release\"\${MAIN_APP}\")
>
> Workaround wrong tool detection using MinGW on Windows:
> if(WIN32ANDNOTMSVC)
> set(GP_TOOL"objdump")
> endif()
> install(CODE "\
> ...
> include(BundleUtilities)
> set(gp_tool\"${GP_TOOL}\")
> fixup_bundle(\"\${MAIN_APP}\"\"\"\"${LIBS_PATH}\")
> ...
> And there is the escaping... You have to escape quotes inside
> the script. Escape the '$' sign if you want to refer to the
> variable at install time. Remember that you don't have access
> to configure-time variables at install time. Unescaped '$' prefix
> will be replaced by its value at configure time, somewhat like
> a macro or a template.
>
> To see the generated script, look into ${CMAKE_BINARY_DIR} with
> the directory name corresponding to the CMakeLists.txt containing
> the install(CODE). Example:
> ${CMAKE_SOURCE_DIR}/cmake/bundle/CMakeLists.txt
> -> ${CMAKE_BINARY_DIR}/cmake/bundle/cmake_install.cmake
> You can check the generated script, its very
> handy when things go wrong.
> Also, it is easier to put all libraries and binaries in their
> own directory. I have this near the begining of my project's
> CMakeLists.
> set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY${CMAKE_BINARY_DIR}/lib)
> set(CMAKE_LIBRARY_OUTPUT_DIRECTORY${CMAKE_BINARY_DIR}/lib)
> set(CMAKE_RUNTIME_OUTPUT_DIRECTORY${CMAKE_BINARY_DIR}/bin)
> If you have some library that you don't want in your bundle,
> you might want to disable install for it, for instance,
> google test:
> add_subdirectory(3rdparty/googletest/EXCLUDE_FROM_ALL)
> I'm using "ntldd.exe -R" on Windows to check the dependencies manually
> (its very much like ldd on Linux). It is much more handy than
> dependency walker.
> Fixup_bundle is not magical either. The library dependencies must be
> known beforehand. It means that if you do some dlopen() tricks at
> runtime, fixup_bundle() cannot know that and you will have to install
> these libraries manually yourself. One notable example of this is
> Intel MKL using the Single Dynamic Library (mkl_rt). Using this
> library entry point simplifies the linking and will load the
> best library at runtime depending on the hardware. Fixup_bundle() will
> copy only mkl_rt and you have to copy the other mkl libraries to
> avoid runtime error.
> Fixup bundle is tricky to put in place, but having a fixed list
> of libraries to copy is even more cumbersome and flaky. When
> supplied with the proper directories, the bundle is right every time.
> Francis
> Le sam. 16 févr. 2019 à 04:04, Andreas Naumann
> <[hidden email] <mailto:[hidden email]>> a écrit :
>
>     Dear CMakers,
>
>     recently I tried to bundle an application in Windows. From the
>     documentation [1] I see that I should provide the directories to the
>     non-system libraries.
>
>     But these information should be already in the properties of the
>     targets, arent they? Is there any extension in cmake, that provides
>     these paths?
>
>     How do other users handle dependencies to external dlls?
>
>
>     [1] https://cmake.org/cmake/help/v3.0/module/BundleUtilities.html
>
>     Regards,
>     Andreas
>
>     --
>
>     Powered by www.kitware.com <http://www.kitware.com>
>
>     Please keep messages on-topic and check the CMake FAQ at:
>     http://www.cmake.org/Wiki/CMake_FAQ
>
>     Kitware offers various services to support the CMake community.
>     For more information on each offering, please visit:
>
>     CMake Support: http://cmake.org/cmake/help/support.html
>     CMake Consulting: http://cmake.org/cmake/help/consulting.html
>     CMake Training Courses: http://cmake.org/cmake/help/training.html
>
>     Visit other Kitware open-source projects at
>     http://www.kitware.com/opensource/opensource.html
>
>     Follow this link to subscribe/unsubscribe:
>     https://cmake.org/mailman/listinfo/cmake
>
> --
> Francis Giraldeau
>

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake