Specifying different compilers for subsets of a project

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Specifying different compilers for subsets of a project

George Zagaris
Dear all,

I am working on a project where on certain platforms, namely, on a BG/Q or Cray, we have to cross-compile the code, but still, there are parts of the code that should only be compiled for the front end, login nodes.

Typically, we have a toolchain file, which among other things sets:
- CMAKE_CXX_COMPILER
- CMAKE_C_COMPILER
- CMAKE_FORTRAN_COMPILER

to the respective cross-compilers available on the target platform.

This compiles everything for the back-end.

The obvious way to compile front-end tools is to create another directory, e.g., "front-end-build" and run cmake therein to re-configure and build with a front-end compiler.

However, since only a relatively small subset of the files needs to be compiled for the front-end, it is desirable to do this within the same build, which leads to my question:

Is it possible to tell CMake to build certain files with a different compiler? I know there is a way to set file properties to control the compiler flags of certain files. Is there a "blessed" approach to control the compiler for certain files and/or directories (and avoid things like set(CMAKE_CXX_COMPILER...) within a CMakeLists file)?

Thank you very much for all your help.

Best,
George

 

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Specifying different compilers for subsets of a project

Marcel Loose
Hi George,

You could (ab)use the "assembler plugin" system for that. Define your own CMake-ASM-BGQ*.cmake files and use the assembler to compile the sources.

Here's how I did this a couple of years ago for the BG/P.

In the CMakeLists.txt file for the files that need to be compiled with the BG/P compiler I have:
## ---------------------------------------------------------------------------
## Enable BGP specific assembler.
## Use the BGP assembler also for linking C/C++ programs.
## ---------------------------------------------------------------------------
enable_language(ASM-BGP)
set(CMAKE_C_LINK_EXECUTABLE ${CMAKE_ASM-BGP_LINK_EXECUTABLE})
set(CMAKE_CXX_LINK_EXECUTABLE ${CMAKE_ASM-BGP_LINK_EXECUTABLE})

CMakeASM-BGPInformation.cmake:
set(ASM_DIALECT "-BGP")
set(CMAKE_ASM${ASM_DIALECT}_SOURCE_FILE_EXTENSIONS S)
set(CMAKE_ASM${ASM_DIALECT}_COMPILE_OBJECT "<CMAKE_ASM${ASM_DIALECT}_COMPILER> <DEFINES> <FLAGS> -c -o <OBJECT> <SOURCE>")
set(CMAKE_ASM${ASM_DIALECT}_LINK_EXECUTABLE "<CMAKE_ASM${ASM_DIALECT}_COMPILER> <FLAGS> <LINK_FLAGS> <OBJECTS>  -o <TARGET> <LINK_LIBRARIES>")
include(CMakeASMInformation)
set(ASM_DIALECT)

CMakeDetermineASM-BGPCompiler.cmake:
set(ASM_DIALECT "-BGP")
set(CMAKE_ASM${ASM_DIALECT}_COMPILER_INIT ${CMAKE_ASM_COMPILER})
include(CMakeDetermineASMCompiler)
set(ASM_DIALECT)

CMakeTestASM-BGPCompiler.cmake:
set(ASM_DIALECT "-BGP")
include(CMakeTestASMCompiler)
set(ASM_DIALECT)

Hope this helps.

Regards,
Marcel Loose.

On 15/10/14 04:23, George Zagaris wrote:
Dear all,

I am working on a project where on certain platforms, namely, on a BG/Q or Cray, we have to cross-compile the code, but still, there are parts of the code that should only be compiled for the front end, login nodes.

Typically, we have a toolchain file, which among other things sets:
- CMAKE_CXX_COMPILER
- CMAKE_C_COMPILER
- CMAKE_FORTRAN_COMPILER

to the respective cross-compilers available on the target platform.

This compiles everything for the back-end.

The obvious way to compile front-end tools is to create another directory, e.g., "front-end-build" and run cmake therein to re-configure and build with a front-end compiler.

However, since only a relatively small subset of the files needs to be compiled for the front-end, it is desirable to do this within the same build, which leads to my question:

Is it possible to tell CMake to build certain files with a different compiler? I know there is a way to set file properties to control the compiler flags of certain files. Is there a "blessed" approach to control the compiler for certain files and/or directories (and avoid things like set(CMAKE_CXX_COMPILER...) within a CMakeLists file)?

Thank you very much for all your help.

Best,
George

 




--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

loose.vcf (304 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Specifying different compilers for subsets of a project

George Zagaris
Hi Marcel,

Thanks for your e-mail. I am not sure if your approach wil fully satisfy my use case. I need to compile on both the back-end and front-end.

My question really boils down to whether it's possible to use two different compilers for the same language within the same configuration.

I was hoping there would be a way to set properties on individual files or directories and specify a different compiler (as one could specify different compilers flags). Is that possible?

Perhaps, another alternative is to define a FRONTEND_C, FRONTEND_CXX and FRONTEND_F90 languages, as well as, define associated "add_frontend_executable()" and "add_frontend_library()", which would work with the frontend compiler when we are cross-compiling, or call "add_executable()" and "add_library" otherwise. This may be a bit of overkill though (but very useful), given that only a relative small subset of the project needs to be compiled for the front-end. 

Is there a good guide on how would one could define a different language in CMake?

I am curious if there are any other known alternatives?

Cheers,
George

On Wed, Oct 15, 2014 at 1:54 AM, Marcel Loose <[hidden email]> wrote:
Hi George,

You could (ab)use the "assembler plugin" system for that. Define your own CMake-ASM-BGQ*.cmake files and use the assembler to compile the sources.

Here's how I did this a couple of years ago for the BG/P.

In the CMakeLists.txt file for the files that need to be compiled with the BG/P compiler I have:
## ---------------------------------------------------------------------------
## Enable BGP specific assembler.
## Use the BGP assembler also for linking C/C++ programs.
## ---------------------------------------------------------------------------
enable_language(ASM-BGP)
set(CMAKE_C_LINK_EXECUTABLE ${CMAKE_ASM-BGP_LINK_EXECUTABLE})
set(CMAKE_CXX_LINK_EXECUTABLE ${CMAKE_ASM-BGP_LINK_EXECUTABLE})

CMakeASM-BGPInformation.cmake:
set(ASM_DIALECT "-BGP")
set(CMAKE_ASM${ASM_DIALECT}_SOURCE_FILE_EXTENSIONS S)
set(CMAKE_ASM${ASM_DIALECT}_COMPILE_OBJECT "<CMAKE_ASM${ASM_DIALECT}_COMPILER> <DEFINES> <FLAGS> -c -o <OBJECT> <SOURCE>")
set(CMAKE_ASM${ASM_DIALECT}_LINK_EXECUTABLE "<CMAKE_ASM${ASM_DIALECT}_COMPILER> <FLAGS> <LINK_FLAGS> <OBJECTS>  -o <TARGET> <LINK_LIBRARIES>")
include(CMakeASMInformation)
set(ASM_DIALECT)

CMakeDetermineASM-BGPCompiler.cmake:
set(ASM_DIALECT "-BGP")
set(CMAKE_ASM${ASM_DIALECT}_COMPILER_INIT ${CMAKE_ASM_COMPILER})
include(CMakeDetermineASMCompiler)
set(ASM_DIALECT)

CMakeTestASM-BGPCompiler.cmake:
set(ASM_DIALECT "-BGP")
include(CMakeTestASMCompiler)
set(ASM_DIALECT)

Hope this helps.

Regards,
Marcel Loose.


On 15/10/14 04:23, George Zagaris wrote:
Dear all,

I am working on a project where on certain platforms, namely, on a BG/Q or Cray, we have to cross-compile the code, but still, there are parts of the code that should only be compiled for the front end, login nodes.

Typically, we have a toolchain file, which among other things sets:
- CMAKE_CXX_COMPILER
- CMAKE_C_COMPILER
- CMAKE_FORTRAN_COMPILER

to the respective cross-compilers available on the target platform.

This compiles everything for the back-end.

The obvious way to compile front-end tools is to create another directory, e.g., "front-end-build" and run cmake therein to re-configure and build with a front-end compiler.

However, since only a relatively small subset of the files needs to be compiled for the front-end, it is desirable to do this within the same build, which leads to my question:

Is it possible to tell CMake to build certain files with a different compiler? I know there is a way to set file properties to control the compiler flags of certain files. Is there a "blessed" approach to control the compiler for certain files and/or directories (and avoid things like set(CMAKE_CXX_COMPILER...) within a CMakeLists file)?

Thank you very much for all your help.

Best,
George

 




--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Specifying different compilers for subsets of a project

Chuck Atkins
In reply to this post by George Zagaris
Hi George,
 
The obvious way to compile front-end tools is to create another directory, e.g., "front-end-build" and run cmake therein to re-configure and build with a front-end compiler.

However, since only a relatively small subset of the files needs to be compiled for the front-end, it is desirable to do this within the same build

This is a common case that occurs when cross compiling where certain targets are used to generate files used in the rest of the build.  Using different build directories is the typical way, but you can reduce the host build as much a possible by creating a BuildTools or CrossTools custom target with depends only on the targets needing to be build on the host system. This way, you can build only the subset needed on the host and in the cross-build, just point the host build directory.  This is how both VTK and ParaView do it.

Trying to specify multiple compilers isn't so straight forward.  Consider the entire platform that goes in to cross compiling: you have the entire build toolset, but then you also have the entire find infrastructure that needs to look in different places for includes and libraries.  There's an entire stack of variables, tools, and settings that are needed to build on a given platform and you would basically need a whole duplicate set generated from a secondary configure run to make it work in the general case.  This is what is effectively happening anyways in the two-builds scenario.

- Chuck


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Specifying different compilers for subsets of a project

George Zagaris
Hi Chuck,

Thanks for your e-mail.

Yes, the different build directory strategy is definitely the clean way to do this.  

However, it is my experience that folks not familiar with CMake tend to get a knee-jerk reaction when required to configure and build seperately. Especially, when something like this is more easily hacked in raw Makefiles, for example. Hence, I am investigating various alternatives to contrast with the nominal separate build directory strategy.

I really like the custom target suggestion -- makes perfect sense.

Best,
George

On Thu, Oct 16, 2014 at 11:32 AM, Chuck Atkins <[hidden email]> wrote:
Hi George,
 
The obvious way to compile front-end tools is to create another directory, e.g., "front-end-build" and run cmake therein to re-configure and build with a front-end compiler.

However, since only a relatively small subset of the files needs to be compiled for the front-end, it is desirable to do this within the same build

This is a common case that occurs when cross compiling where certain targets are used to generate files used in the rest of the build.  Using different build directories is the typical way, but you can reduce the host build as much a possible by creating a BuildTools or CrossTools custom target with depends only on the targets needing to be build on the host system. This way, you can build only the subset needed on the host and in the cross-build, just point the host build directory.  This is how both VTK and ParaView do it.

Trying to specify multiple compilers isn't so straight forward.  Consider the entire platform that goes in to cross compiling: you have the entire build toolset, but then you also have the entire find infrastructure that needs to look in different places for includes and libraries.  There's an entire stack of variables, tools, and settings that are needed to build on a given platform and you would basically need a whole duplicate set generated from a secondary configure run to make it work in the general case.  This is what is effectively happening anyways in the two-builds scenario.

- Chuck



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake