Cannot import a project twice when cross compiling (as host-tool and for the target arch)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Cannot import a project twice when cross compiling (as host-tool and for the target arch)

Settenvini, Matteo
Dear all,

I have the following problem which I don't know how to best solve.
Any help is appreciated.

============== Use Case ==============

Consider the following use case:

* We are cross-compiling.
* A certain project A depends on project B in two different ways:
  - it uses an executable target from B to generate some files, e.g. a compiler.
  - one of its libraries links against a library of B.

We would like to be able to do (not possible with CMake at the moment):


  find_package(B REQUIRED)
  # use B::lib normally. When crosscompiling, it will be the one compiled for the target.

  find_package(B REQUIRED NO_CMAKE_FIND_ROOT_PATH)
  # use B::exe knowing that it will always run from the host system.


This is for instance a use case we have for flatbuffers / protobuf.

IMPORTED and ALIAS targets should not be global, or it should be
possible to override them. Or else, it should be possible to
dynamically specify a namespace when importing targets through
find_package().

I opened a bug at https://gitlab.kitware.com/cmake/cmake/issues/18169,
but it got closed as there are no plans to change the way CMake works.

The issue contains also a test case, if you'd like to try it out.

============== Problem ==============

The user cannot import a project twice to get the appropriate targets
when cross-compiling: once for the host architecture and once for the
target architecture.

Right now, the user needs to manually use e.g. find_program() instead
of relying on installed locations, which can be annoying if the
binaries are installed outside the environment PATH. This can be
worked around, but we would like to use targets in place of
find_program() or find_library() calls.

The problem is much more apparent in big superbuilds where you have
hundreds of projects brought together in one build and no control over
components outside your own. It is too hard to ensure a stable
configuration order (so that find_package() is always called for the
host version before the target version, or vice-versa).

============== Help needed ==============

How would you solve the issue, so that different targets are available
for both the version built for the host and target architectures?

Note that sometimes these external projects are not under our control,
and they come directly from upstream as git submodules. So we would
like to touch them as little as possible.

Bests,
Matteo Settenvini
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Cannot import a project twice when cross compiling (as host-tool and for the target arch)

Craig Scott-3
Not quite sure if it fits your use case, but we use a technique at work basically as described in the following:


Your case sounds more complicated, but maybe the above technique might help if you can organise things that way.



On Wed, Jul 11, 2018 at 10:54 PM, Settenvini, Matteo <[hidden email]> wrote:
Dear all,

I have the following problem which I don't know how to best solve.
Any help is appreciated.

============== Use Case ==============

Consider the following use case:

* We are cross-compiling.
* A certain project A depends on project B in two different ways:
  - it uses an executable target from B to generate some files, e.g. a compiler.
  - one of its libraries links against a library of B.

We would like to be able to do (not possible with CMake at the moment):


  find_package(B REQUIRED)
  # use B::lib normally. When crosscompiling, it will be the one compiled for the target.

  find_package(B REQUIRED NO_CMAKE_FIND_ROOT_PATH)
  # use B::exe knowing that it will always run from the host system.


This is for instance a use case we have for flatbuffers / protobuf.

IMPORTED and ALIAS targets should not be global, or it should be
possible to override them. Or else, it should be possible to
dynamically specify a namespace when importing targets through
find_package().

I opened a bug at https://gitlab.kitware.com/cmake/cmake/issues/18169,
but it got closed as there are no plans to change the way CMake works.

The issue contains also a test case, if you'd like to try it out.

============== Problem ==============

The user cannot import a project twice to get the appropriate targets
when cross-compiling: once for the host architecture and once for the
target architecture.

Right now, the user needs to manually use e.g. find_program() instead
of relying on installed locations, which can be annoying if the
binaries are installed outside the environment PATH. This can be
worked around, but we would like to use targets in place of
find_program() or find_library() calls.

The problem is much more apparent in big superbuilds where you have
hundreds of projects brought together in one build and no control over
components outside your own. It is too hard to ensure a stable
configuration order (so that find_package() is always called for the
host version before the target version, or vice-versa).

============== Help needed ==============

How would you solve the issue, so that different targets are available
for both the version built for the host and target architectures?

Note that sometimes these external projects are not under our control,
and they come directly from upstream as git submodules. So we would
like to touch them as little as possible.

Bests,
Matteo Settenvini
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake



--
Craig Scott
Melbourne, Australia

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Cannot import a project twice when cross compiling (as host-tool and for the target arch)

Settenvini, Matteo

Thanks Craig,


we are already doing roughly that, plus some custom logic we built around options to be able to forward them to the second CMake process.

It's a bit more complex for us, but the concepts are the same.


The problem is just being able to use targets for the version of the executables compiled for the host, instead of using find_program or hardcoding paths like you do at step (e) in your StackOverflow answer.


It wouldn't be that important if all people using our internal CMake API were also CMake experts, but unfortunately we have lots of baffled developers by having to call "find_program()" and then variables, instead of "find_package(pkg REQUIRED NO_CMAKE_FIND_ROOT_PATH)" and then use an imported target. Also, this is asymmetrical when non-crosscompiling, which adds more logic than necessary in several places.


Cheers,

Matteo Settenvini



From: Craig Scott <[hidden email]>
Sent: Thursday, July 12, 2018 12:00:18 AM
To: Settenvini, Matteo
Cc: [hidden email]
Subject: Re: [CMake] Cannot import a project twice when cross compiling (as host-tool and for the target arch)
 
Not quite sure if it fits your use case, but we use a technique at work basically as described in the following:


Your case sounds more complicated, but maybe the above technique might help if you can organise things that way.



On Wed, Jul 11, 2018 at 10:54 PM, Settenvini, Matteo <[hidden email]> wrote:
Dear all,

I have the following problem which I don't know how to best solve.
Any help is appreciated.

============== Use Case ==============

Consider the following use case:

* We are cross-compiling.
* A certain project A depends on project B in two different ways:
  - it uses an executable target from B to generate some files, e.g. a compiler.
  - one of its libraries links against a library of B.

We would like to be able to do (not possible with CMake at the moment):


  find_package(B REQUIRED)
  # use B::lib normally. When crosscompiling, it will be the one compiled for the target.

  find_package(B REQUIRED NO_CMAKE_FIND_ROOT_PATH)
  # use B::exe knowing that it will always run from the host system.


This is for instance a use case we have for flatbuffers / protobuf.

IMPORTED and ALIAS targets should not be global, or it should be
possible to override them. Or else, it should be possible to
dynamically specify a namespace when importing targets through
find_package().

I opened a bug at https://gitlab.kitware.com/cmake/cmake/issues/18169,
but it got closed as there are no plans to change the way CMake works.

The issue contains also a test case, if you'd like to try it out.

============== Problem ==============

The user cannot import a project twice to get the appropriate targets
when cross-compiling: once for the host architecture and once for the
target architecture.

Right now, the user needs to manually use e.g. find_program() instead
of relying on installed locations, which can be annoying if the
binaries are installed outside the environment PATH. This can be
worked around, but we would like to use targets in place of
find_program() or find_library() calls.

The problem is much more apparent in big superbuilds where you have
hundreds of projects brought together in one build and no control over
components outside your own. It is too hard to ensure a stable
configuration order (so that find_package() is always called for the
host version before the target version, or vice-versa).

============== Help needed ==============

How would you solve the issue, so that different targets are available
for both the version built for the host and target architectures?

Note that sometimes these external projects are not under our control,
and they come directly from upstream as git submodules. So we would
like to touch them as little as possible.

Bests,
Matteo Settenvini
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake



--
Craig Scott
Melbourne, Australia

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake