Consuming results of ExternalProject_Add

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Consuming results of ExternalProject_Add

Andrey Pokrovskiy
Hi,

I'm using ExternalProject_Add() to build OpenSLL library. After
install step I have following artifacts available:
* libcrypto.a
* libssl.a
* include/openssl/*.h

I wonder, what is the "canonical" way to make those artifacts
available for other targets in the project?

FindPackage style thing with OPENSSL_LIBRARIES and OPENSSL_INCLUDE_DIR
variables will not work, because OpenSSL target and executable that
uses it are on the same level (I think using CACHE to propagate
variable value up is a dirty hack):
* src/openssl/CMakeLists.txt
* src/my_executable/CMakeLists.txt

Ideally I would like to use add_library() with target_include_directories() and
target_link_libraries(). But the only way to do that is to use
IMPORTED libraries, like that:

add_library(crypto STATIC IMPORTED GLOBAL)
add_dependencies(crypto openssl_external_project)
set_property(TARGET crypto PROPERTY INTERFACE_LINK_LIBRARIES
${OPENSSL_PREFIX}/lib/libcrypto.a)
set_property(TARGET crypto PROPERTY INTERFACE_INCLUDE_DIRECTORIES
${OPENSSL_PREFIX}/include)

But then I get an error:
$ cmake ..

CMake Error in src/my_executable/CMakeLists.txt:

  Imported target "crypto" includes non-existent path

 "<...>/build.dir/src/openssl/openssl-1.0.2.install/include"

Obviously, that happens because include directory was not created yet
(it will after make).

I understand that from general CMake standpoint - IMPORTED libraries
are something that already exists and not part of the build. But then
I don't understand how this ExternalProject_Add thing is supposed to
be used.

Any ideas what I'm doing wrong?
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Consuming results of ExternalProject_Add

Jean-Christophe Fillion-Robin
Hi Andrey,

Since there is already a FindOpenSSL.cmake module [1], configuring the consumer project with the variable expected by the FindOpenSSL.cmake module is the easiest.

See https://github.com/Slicer/Slicer/blob/ee84fba523d85be3539bf8c83c8eb73d4f59cfff/SuperBuild/External_OpenSSL.cmake#L231-L239


[1] https://github.com/Kitware/CMake/blob/master/Modules/FindOpenSSL.cmake

Hth
Jc

On Tue, Feb 17, 2015 at 3:27 PM, Andrey Pokrovskiy <[hidden email]> wrote:
Hi,

I'm using ExternalProject_Add() to build OpenSLL library. After
install step I have following artifacts available:
* libcrypto.a
* libssl.a
* include/openssl/*.h

I wonder, what is the "canonical" way to make those artifacts
available for other targets in the project?

FindPackage style thing with OPENSSL_LIBRARIES and OPENSSL_INCLUDE_DIR
variables will not work, because OpenSSL target and executable that
uses it are on the same level (I think using CACHE to propagate
variable value up is a dirty hack):
* src/openssl/CMakeLists.txt
* src/my_executable/CMakeLists.txt

Ideally I would like to use add_library() with target_include_directories() and
target_link_libraries(). But the only way to do that is to use
IMPORTED libraries, like that:

add_library(crypto STATIC IMPORTED GLOBAL)
add_dependencies(crypto openssl_external_project)
set_property(TARGET crypto PROPERTY INTERFACE_LINK_LIBRARIES
${OPENSSL_PREFIX}/lib/libcrypto.a)
set_property(TARGET crypto PROPERTY INTERFACE_INCLUDE_DIRECTORIES
${OPENSSL_PREFIX}/include)

But then I get an error:
$ cmake ..

CMake Error in src/my_executable/CMakeLists.txt:

  Imported target "crypto" includes non-existent path

 "<...>/build.dir/src/openssl/openssl-1.0.2.install/include"

Obviously, that happens because include directory was not created yet
(it will after make).

I understand that from general CMake standpoint - IMPORTED libraries
are something that already exists and not part of the build. But then
I don't understand how this ExternalProject_Add thing is supposed to
be used.

Any ideas what I'm doing wrong?
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--
+1 919 869 8849

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Consuming results of ExternalProject_Add

Andrey Pokrovskiy
Thanks for your reply, Jean-Christophe.

I don't see how External_OpenSSL.cmake is used in Slicer (probably
some weird scheme is involved).
But from what I see, it supposed to be used like that:

  include(External_OpenSSL)
  add_executable(my_executable main.cpp ...)
  target_include_directories(my_executable ${OPENSSL_INCLUDE_DIR})
  target_link_libraries(my_executable ${OPENSSL_LIBRARIES})

That means that I will need to include External_OpenSSL in every
CMakeLists.txt that uses OpenSSL. That doesn't sounds right to me.

And if not, then something adds OPENSSL_INCLUDE_DIR and
OPENSSL_LIBRARIES to CACHE which is also doesn't sounds right.

Also this will not setup build dependency between target and OpenSSL.
I know, I could use additional add_dependencies() for that, but that's
clumsy. 3 lines instead of just:

  target_link_libraries(my_executable crypto ssl)

CMake already has a concept of "libraries" (added with add_library).
There should be a way of saying "Hey, this static library comes from
that external project. It also requires such and such include
directories".

On Tue, Feb 17, 2015 at 1:53 PM, Jean-Christophe Fillion-Robin
<[hidden email]> wrote:

> Hi Andrey,
>
> Since there is already a FindOpenSSL.cmake module [1], configuring the
> consumer project with the variable expected by the FindOpenSSL.cmake module
> is the easiest.
>
> See
> https://github.com/Slicer/Slicer/blob/ee84fba523d85be3539bf8c83c8eb73d4f59cfff/SuperBuild/External_OpenSSL.cmake#L231-L239
>
>
> [1] https://github.com/Kitware/CMake/blob/master/Modules/FindOpenSSL.cmake
>
> Hth
> Jc
>
> On Tue, Feb 17, 2015 at 3:27 PM, Andrey Pokrovskiy <[hidden email]>
> wrote:
>>
>> Hi,
>>
>> I'm using ExternalProject_Add() to build OpenSLL library. After
>> install step I have following artifacts available:
>> * libcrypto.a
>> * libssl.a
>> * include/openssl/*.h
>>
>> I wonder, what is the "canonical" way to make those artifacts
>> available for other targets in the project?
>>
>> FindPackage style thing with OPENSSL_LIBRARIES and OPENSSL_INCLUDE_DIR
>> variables will not work, because OpenSSL target and executable that
>> uses it are on the same level (I think using CACHE to propagate
>> variable value up is a dirty hack):
>> * src/openssl/CMakeLists.txt
>> * src/my_executable/CMakeLists.txt
>>
>> Ideally I would like to use add_library() with
>> target_include_directories() and
>> target_link_libraries(). But the only way to do that is to use
>> IMPORTED libraries, like that:
>>
>> add_library(crypto STATIC IMPORTED GLOBAL)
>> add_dependencies(crypto openssl_external_project)
>> set_property(TARGET crypto PROPERTY INTERFACE_LINK_LIBRARIES
>> ${OPENSSL_PREFIX}/lib/libcrypto.a)
>> set_property(TARGET crypto PROPERTY INTERFACE_INCLUDE_DIRECTORIES
>> ${OPENSSL_PREFIX}/include)
>>
>> But then I get an error:
>> $ cmake ..
>>
>> CMake Error in src/my_executable/CMakeLists.txt:
>>
>>   Imported target "crypto" includes non-existent path
>>
>>  "<...>/build.dir/src/openssl/openssl-1.0.2.install/include"
>>
>> Obviously, that happens because include directory was not created yet
>> (it will after make).
>>
>> I understand that from general CMake standpoint - IMPORTED libraries
>> are something that already exists and not part of the build. But then
>> I don't understand how this ExternalProject_Add thing is supposed to
>> be used.
>>
>> Any ideas what I'm doing wrong?
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake
>
>
>
>
> --
> +1 919 869 8849
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Consuming results of ExternalProject_Add

Jean-Christophe Fillion-Robin
Hi Andrey,

On Tue, Feb 17, 2015 at 5:52 PM, Andrey Pokrovskiy <[hidden email]> wrote:

I don't see how External_OpenSSL.cmake is used in Slicer (probably
some weird scheme is involved).

We used "Artichoke" that provides a set of convenience function to managed "superbuild" based project.
See [1] and [2] for mode details.

[1] http://public.kitware.com/pipermail/cmake/2014-June/057735.html
[2] https://github.com/commontk/Artichoke

 
But from what I see, it supposed to be used like that:

  include(External_OpenSSL)
  add_executable(my_executable main.cpp ...)
  target_include_directories(my_executable ${OPENSSL_INCLUDE_DIR})
  target_link_libraries(my_executable ${OPENSSL_LIBRARIES})

That means that I will need to include External_OpenSSL in every
CMakeLists.txt that uses OpenSSL. That doesn't sounds right to me.


One way of using external projects is to consider it automate the following steps:

 (1) manual build of OpenSSL
 (2) configuration of your project Foo passing either OPENSSL_CRYPTO_LIBRARY/OPENSSL_SSL_LIBRARY options on unix or LIB_EAY_DEBUG/LIB_EAY_RELEASE/SSL_EAY_DEBUG/SSL_EAY_RELEASE on windows, and then build of the project


And since there are no need to manually rebuild openssl for each CMakeLists.txt in your project Foo, there are no need to include "External_OpenSSL" in every CMakeLists.txt.

Reading [3] should also be helpful to understand the concepts.

[3] http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html





 

And if not, then something adds OPENSSL_INCLUDE_DIR and
OPENSSL_LIBRARIES to CACHE which is also doesn't sounds right.

Also this will not setup build dependency between target and OpenSSL.
I know, I could use additional add_dependencies() for that, but that's
clumsy. 3 lines instead of just:

  target_link_libraries(my_executable crypto ssl)

After doing:

  find_package(OpenSSL)

in your project, you could then do:

  target_link_libraries(my_executable ${OPENSSL_LIBRARIES})


 

CMake already has a concept of "libraries" (added with add_library).
There should be a way of saying "Hey, this static library comes from
that external project. It also requires such and such include
directories".

In fact, all FindXXXX.cmake module are slowly been converted to create CMake imported targets.

Since the current FindOpenSSL.cmake does not create imported targets, it could be updated to do so. For an example, see [4]

[4] http://www.cmake.org/cmake/help/v3.1/manual/cmake-developer.7.html#a-sample-find-module
[5] https://github.com/Kitware/CMake/commit/bcb0e38


Hth
Jc




--
+1 919 869 8849

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Consuming results of ExternalProject_Add

Andrey Pokrovskiy
Thanks for clarification, JC. The links you provided are a good read.

However, I would like to avoid "super-build" technique. It needlessly
(in my case) complicates and obfuscates things.

> In fact, all FindXXXX.cmake module are slowly been converted to create CMake imported targets.

FindXXXX.cmake modules assume that artifacts (libraries, headers)
already exist and use INTERFACE or/and IMPORTED targets. But there are
problems when headers and libraries generated with the build. See
thread about that specific limitation [1].

* add_library(xxx INTERFACE) - target_link_libraries and
target_include_directories can be used but library xxx can't depend on
anything (but it must depend on external project that will generate
it)
* add_library(xxx IMPORTED GLOBAL) - INTERFACE_INCLUDE_DIRECTORIES and
INTERFACE_LINK_LIBRARIES can be used, but will produce error during
configuration that include dir doesn't exist and associated libraries
will be "_NOT_FOUND".

As for now, I ended up with the following:

  ExternalProject_Add(openssl ...)
  add_library(crypto INTERFACE)
  target_link_libraries(crypto INTERFACE
${OPENSSL_INSTALL_PREFIX}/lib/libcrypto.a)
  target_include_directories(crypto INTERFACE ${OPENSSL_INSTALL_PREFIX}/include)
  add_library(ssl INTERFACE)
  target_link_libraries(ssl INTERFACE ${OPENSSL_INSTALL_PREFIX}/lib/libssl.a)
  target_include_directories(ssl INTERFACE ${OPENSSL_INSTALL_PREFIX}/include)

With the requirement to add_dependencies(... openssl) when target
depends on ssl or crypto libraries, like that:

  add_executable(my_executable ${SOURCES})
  add_dependencies(my_executable openssl)
  target_link_libraries(my_executable crypto ssl)

Quite inconsistent behaviour if you ask me.

Thanks, anyway. I will keep in mind this "super-build" thing in case
my solution will not scale too well.

[1] http://www.cmake.org/pipermail/cmake/2015-February/059885.html

On Tue, Feb 17, 2015 at 9:58 PM, Jean-Christophe Fillion-Robin
<[hidden email]> wrote:

> Hi Andrey,
>
> On Tue, Feb 17, 2015 at 5:52 PM, Andrey Pokrovskiy <[hidden email]>
> wrote:
>>
>>
>> I don't see how External_OpenSSL.cmake is used in Slicer (probably
>> some weird scheme is involved).
>
>
> We used "Artichoke" that provides a set of convenience function to managed
> "superbuild" based project.
> See [1] and [2] for mode details.
>
> [1] http://public.kitware.com/pipermail/cmake/2014-June/057735.html
> [2] https://github.com/commontk/Artichoke
>
>
>>
>> But from what I see, it supposed to be used like that:
>>
>>   include(External_OpenSSL)
>>   add_executable(my_executable main.cpp ...)
>>   target_include_directories(my_executable ${OPENSSL_INCLUDE_DIR})
>>   target_link_libraries(my_executable ${OPENSSL_LIBRARIES})
>>
>> That means that I will need to include External_OpenSSL in every
>> CMakeLists.txt that uses OpenSSL. That doesn't sounds right to me.
>
>
>
> One way of using external projects is to consider it automate the following
> steps:
>
>  (1) manual build of OpenSSL
>  (2) configuration of your project Foo passing either
> OPENSSL_CRYPTO_LIBRARY/OPENSSL_SSL_LIBRARY options on unix or
> LIB_EAY_DEBUG/LIB_EAY_RELEASE/SSL_EAY_DEBUG/SSL_EAY_RELEASE on windows, and
> then build of the project
>
>
> And since there are no need to manually rebuild openssl for each
> CMakeLists.txt in your project Foo, there are no need to include
> "External_OpenSSL" in every CMakeLists.txt.
>
> Reading [3] should also be helpful to understand the concepts.
>
> [3]
> http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html
>
>
>
>
>
>
>>
>>
>> And if not, then something adds OPENSSL_INCLUDE_DIR and
>> OPENSSL_LIBRARIES to CACHE which is also doesn't sounds right.
>>
>> Also this will not setup build dependency between target and OpenSSL.
>> I know, I could use additional add_dependencies() for that, but that's
>> clumsy. 3 lines instead of just:
>>
>>   target_link_libraries(my_executable crypto ssl)
>
>
> After doing:
>
>   find_package(OpenSSL)
>
> in your project, you could then do:
>
>   target_link_libraries(my_executable ${OPENSSL_LIBRARIES})
>
>
>
>>
>>
>> CMake already has a concept of "libraries" (added with add_library).
>> There should be a way of saying "Hey, this static library comes from
>> that external project. It also requires such and such include
>> directories".
>
>
> In fact, all FindXXXX.cmake module are slowly been converted to create CMake
> imported targets.
>
> Since the current FindOpenSSL.cmake does not create imported targets, it
> could be updated to do so. For an example, see [4]
>
> [4]
> http://www.cmake.org/cmake/help/v3.1/manual/cmake-developer.7.html#a-sample-find-module
> [5] https://github.com/Kitware/CMake/commit/bcb0e38
>
>
> Hth
> Jc
>
>
>
>
> --
> +1 919 869 8849
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake