FindMPI.cmake in 3.11.4

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

FindMPI.cmake in 3.11.4

Burlen Loring
Hi,

I'm trying to transition from cmake 3.8.2 to cmake 3.11.4. There are a
number of changes the way FindMPI.cmake behaves in 3.11 that are making
this difficult.

In 3.11 how do I override the settings? For instance I want to set
MPI_C_LIBRARIES, MPI_C_INCLUDE_DIRS? In 3.8 all I had to do was set
those variables myself, before find_package(MPI), and my setting were
used. in 3.11 find_package(MPI) ignores, and wipes out my settings, and
then fails.

Isn't it a defacto standard of cmake to be able to override find modules
by setting the libraries and include dirs before the module runs?

Burlen


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindMPI.cmake in 3.11.4

Chuck Atkins
Hi Burlen,
_LIBRARIES and _INCLUDE_DIRS are non-cached output variables.  The new functionality should be documented as to how to guide the current FindMPI: https://cmake.org/cmake/help/v3.12/module/FindMPI.html#variables-for-locating-mpi .  That being said, is it incorrectl6y detecting or determine the MPI installation?  If that's the case then we should probably fix that issue instead.

----------
Chuck Atkins
Staff R&D Engineer, Scientific Computing
Kitware, Inc.
(518) 881-1183


On Thu, Sep 13, 2018 at 4:46 PM Burlen Loring <[hidden email]> wrote:
Hi,

I'm trying to transition from cmake 3.8.2 to cmake 3.11.4. There are a
number of changes the way FindMPI.cmake behaves in 3.11 that are making
this difficult.

In 3.11 how do I override the settings? For instance I want to set
MPI_C_LIBRARIES, MPI_C_INCLUDE_DIRS? In 3.8 all I had to do was set
those variables myself, before find_package(MPI), and my setting were
used. in 3.11 find_package(MPI) ignores, and wipes out my settings, and
then fails.

Isn't it a defacto standard of cmake to be able to override find modules
by setting the libraries and include dirs before the module runs?

Burlen


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindMPI.cmake in 3.11.4

Burlen Loring
Of course I read the documentation. Alas after trying various settings described there in, nothing worked. my settings are ignored and erased.

why did they MPI_C_LIBRARIES and MPI_C_INCLUDE_DIRS/PATH get changed from cache variables to not chached variables? that seems to be the cause of the issue.  Now how does one override the detection and tell it what to use in the new module?

Burlen

On 09/14/2018 10:32 AM, Chuck Atkins wrote:
Hi Burlen,
_LIBRARIES and _INCLUDE_DIRS are non-cached output variables.  The new functionality should be documented as to how to guide the current FindMPI: https://cmake.org/cmake/help/v3.12/module/FindMPI.html#variables-for-locating-mpi .  That being said, is it incorrectl6y detecting or determine the MPI installation?  If that's the case then we should probably fix that issue instead.

----------
Chuck Atkins
Staff R&D Engineer, Scientific Computing
Kitware, Inc.
(518) 881-1183


On Thu, Sep 13, 2018 at 4:46 PM Burlen Loring <[hidden email]> wrote:
Hi,

I'm trying to transition from cmake 3.8.2 to cmake 3.11.4. There are a
number of changes the way FindMPI.cmake behaves in 3.11 that are making
this difficult.

In 3.11 how do I override the settings? For instance I want to set
MPI_C_LIBRARIES, MPI_C_INCLUDE_DIRS? In 3.8 all I had to do was set
those variables myself, before find_package(MPI), and my setting were
used. in 3.11 find_package(MPI) ignores, and wipes out my settings, and
then fails.

Isn't it a defacto standard of cmake to be able to override find modules
by setting the libraries and include dirs before the module runs?

Burlen


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindMPI.cmake in 3.11.4

Mateusz Loskot
On Fri, 14 Sep 2018 at 20:57, Burlen Loring <[hidden email]> wrote:
> why did they MPI_C_LIBRARIES and MPI_C_INCLUDE_DIRS/PATH get changed from cache variables to not chached variables?

Possibly, that change was due to clean up to correct the module
according to the actual CMake conventions

https://cmake.org/cmake/help/latest/manual/cmake-developer.7.html#standard-variable-names

"""
Xxx_INCLUDE_DIRS
  The final set of include directories listed in one variable for use
by client code. This should not be a cache entry.

Xxx_LIBRARIES
  The libraries to link against to use Xxx. These should include full
paths. This should not be a cache entry.
"""

Also, find_package_handle_standard_args description also gives similar
hint on the naming:

https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
"""
Therefore these should typically be cache entries such as FOO_LIBRARY
and not output variables like FOO_LIBRARIES.
"""

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Mateusz Loskot
http://mateusz.loskot.net
Reply | Threaded
Open this post in threaded view
|

Re: FindMPI.cmake in 3.11.4

Chuck Atkins
In reply to this post by Burlen Loring
Hi Burlen

Of course I read the documentation.

I certainly didn't mean for that to come off as an "rtfm", so my appologies if it did, that wasn't my intent.

 
why did they MPI_C_LIBRARIES and MPI_C_INCLUDE_DIRS/PATH get changed from cache variables to not chached variables? that seems to be the cause of the issue.

There was a completere-write of FindMPI from scratch to address a wide range of issues with it.  In general many of the "old" behaviors were in opposition to other established "best practices" so it was re-written with many of these in mind.  As such, much of the old behavior w.r.t. variable names may not have been preserved in an attemnt to bring the behavior more inline with recommended current CMake practices.  So in some scenarios it should be considdered a breaking change.


Alas after trying various settings described there in, nothing worked. my settings are ignored and erased.
 
  Now how does one override the detection and tell it what to use in the new module?

There's several different scenarios to try to detect MPI and I believe they happen in this order:
  1. Using the mpicc compiler wrapper as your "regular compiler"
    • libraries and includes should remain empty as nothing needs to be added for MPI to work
    • This is the case when using the CrayPE wrappers
  2. Locating mpicc in your path and interrogating it
    • Determined the set of includes and libraries to be added to the regular compiler to make MPI work
    • This is the typical behavior that get's used in *most* mpi environments.
  3. Falling back to manually searching for headers and libraries
Can you describe your environment and scenario more specifically so I can try to reproduce it?

Thanks
- Chuck


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindMPI.cmake in 3.11.4

Burlen Loring
Hi Chuck,

I have a an install of mpi that has no compiler wrappers. I know the include dir and the list of libraries to use. How do I tell the new cmake module? Is it no longer supported to tell the module what to use for these?

Burlen

On 09/17/2018 06:46 AM, Chuck Atkins wrote:
Hi Burlen

Of course I read the documentation.

I certainly didn't mean for that to come off as an "rtfm", so my appologies if it did, that wasn't my intent.

 
why did they MPI_C_LIBRARIES and MPI_C_INCLUDE_DIRS/PATH get changed from cache variables to not chached variables? that seems to be the cause of the issue.

There was a completere-write of FindMPI from scratch to address a wide range of issues with it.  In general many of the "old" behaviors were in opposition to other established "best practices" so it was re-written with many of these in mind.  As such, much of the old behavior w.r.t. variable names may not have been preserved in an attemnt to bring the behavior more inline with recommended current CMake practices.  So in some scenarios it should be considdered a breaking change.


Alas after trying various settings described there in, nothing worked. my settings are ignored and erased.
 
  Now how does one override the detection and tell it what to use in the new module?

There's several different scenarios to try to detect MPI and I believe they happen in this order:
  1. Using the mpicc compiler wrapper as your "regular compiler"
    • libraries and includes should remain empty as nothing needs to be added for MPI to work
    • This is the case when using the CrayPE wrappers
  2. Locating mpicc in your path and interrogating it
    • Determined the set of includes and libraries to be added to the regular compiler to make MPI work
    • This is the typical behavior that get's used in *most* mpi environments.
  3. Falling back to manually searching for headers and libraries
Can you describe your environment and scenario more specifically so I can try to reproduce it?

Thanks
- Chuck



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindMPI.cmake in 3.11.4

Burlen Loring
In reply to this post by Mateusz Loskot
Thanks, I did not know about the guidelines. I guess my misconception is
based on having seen historically cache variables used in a number of
find modules and also in practice being able to override the module when
needed. So if the outputs of the find module aren't cache variables, how
does one override the module and tell it what to use?

On 09/14/2018 12:30 PM, Mateusz Loskot wrote:

> On Fri, 14 Sep 2018 at 20:57, Burlen Loring <[hidden email]> wrote:
>> why did they MPI_C_LIBRARIES and MPI_C_INCLUDE_DIRS/PATH get changed from cache variables to not chached variables?
> Possibly, that change was due to clean up to correct the module
> according to the actual CMake conventions
>
> https://cmake.org/cmake/help/latest/manual/cmake-developer.7.html#standard-variable-names
>
> """
> Xxx_INCLUDE_DIRS
>    The final set of include directories listed in one variable for use
> by client code. This should not be a cache entry.
>
> Xxx_LIBRARIES
>    The libraries to link against to use Xxx. These should include full
> paths. This should not be a cache entry.
> """
>
> Also, find_package_handle_standard_args description also gives similar
> hint on the naming:
>
> https://cmake.org/cmake/help/latest/module/FindPackageHandleStandardArgs.html
> """
> Therefore these should typically be cache entries such as FOO_LIBRARY
> and not output variables like FOO_LIBRARIES.
> """
>
> Best regards,

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake