Experiments in CMake support for Clang (header & standard) modules

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Experiments in CMake support for Clang (header & standard) modules

David Blaikie
Hi there,

I'm experimenting with creating examples (& potential changes to CMake itself, if needed/useful) of building clang modules (currently using the semi-backwards compatible "header modules", with the intent of also/moving towards supporting pre-standard C++ modules in development in Clang).

The basic commands required are:

  clang++ -fmodules -xc++ -Xclang -emit-module -Xclang -fmodules-codegen -fmodule-name=foo foo.modulemap -o foo.pcm
  clang++ -fmodules -c -fmodule-file=foo.pcm use.cpp
  clang++ -c foo.pcm
  clang++ foo.o use.o -o a.out

My current very simplistic prototype, to build a module file, its respective module object file, and include those in the library/link for anything that depends on this library:

  add_custom_command(
          COMMAND ${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS} -xc++ -c -Xclang -emit-module -fmodules -fmodule-name=Hello ${CMAKE_CURRENT_SOURCE_DIR}/module.modulemap -o ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm -Xclang -fmodules-codegen
          DEPENDS module.modulemap hello.h
          OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm
          COMMENT "Generating hello_module.pcm"
  )
  add_library (Hello hello.cxx ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm)
  target_include_directories(Hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
  target_compile_options(Hello PUBLIC -fmodules -Xclang -fmodule-file=${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm)

(this is based on the example in the CMake docs using Hello/Demo)

This also required one modification to CMake itself to classify a pcm file as a C++ file that needs to be compiled (to execute the 3rd line in the basic command list shown above).

But this isn't ideal - I don't /think/ I've got the dependencies quite right & things might not be rebuilding at the right times.
Also it involves hardcoding a bunch of things like the pcm file names, header files, etc.

Ideally, at least for a simplistic build, I wouldn't mind generating a modulemap from all the .h files (& have those headers listed in the add_library command - perhaps splitting public and private headers in some way, only including the public headers in the module file, likely). Eventually for the standards-proposal version, it's expected that there won't be any modulemap file, but maybe all headers are included in the module compilation (just pass the headers directly to the compiler).

This also doesn't start to approach the issue of how to build modules for external libraries - which I'm happy to discuss/prototype too, though interested in working to streamline the inter-library but intra-project (not inter-project) case first.

Stephen - I saw you were asking some questions about this here ( https://groups.google.com/a/isocpp.org/forum/#!topic/modules/sDIYoU8Uljw & https://github.com/steveire/ModulesExperiments - didn't really understand how this example applied/worked, though - I guess maybe it's a prototype syntax proposal?)

Basically: What do folks think about supporting these sort of features in CMake C++ Builds? Any pointers on how I might best implement this with or without changes to CMake?

Thanks a bunch,
- Dave

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Experiments in CMake support for Clang (header & standard) modules

Stephen Kelly-2
On 04/20/2018 01:39 AM, David Blaikie wrote:
Hi there,

I'm experimenting with creating examples (& potential changes to CMake itself, if needed/useful) of building clang modules (currently using the semi-backwards compatible "header modules", with the intent of also/moving towards supporting pre-standard C++ modules in development in Clang).

Great! Thanks for reaching out. Sorry it has taken me a while to respond. Have you had other response off-list?

The basic commands required are:

  clang++ -fmodules -xc++ -Xclang -emit-module -Xclang -fmodules-codegen -fmodule-name=foo foo.modulemap -o foo.pcm
  clang++ -fmodules -c -fmodule-file=foo.pcm use.cpp
  clang++ -c foo.pcm
  clang++ foo.o use.o -o a.out

Ok. Fundamentally, I am suspicious of having to have a -fmodule-file=foo.pcm for every 'import foo' in each cpp file. I shouldn't have to manually add that each time I add a new import to my cpp file. Even if it can be automated (eg by CMake), I shouldn't have to have my buildsystem be regenerated each time I add an import to my cpp file either.

That's something I mentioned in the google groups post I made which you linked to. How will that work when using Qt or any other library?

Today, a beginner can find a random C++ book, type in a code example from chapter one and put `g++ -I/opt/book_examples prog1.cpp` into a terminal and get something compiling and running. With modules, they'll potentially have to pass a whole list of module files too.

Lots of people manually maintain Makefile-based buildsystems today, and most other companies I've been inside of have their own custom tool or bunch of python scripts, or both. Manually changing such buildsystems to add -fmodule-file or -fmodule-map-file each time an import is added is a significant barrier.

Will my project have to compile the modules files for all of my dependencies? Even more complication for my buildsystem. Do I have to wait for my dependencies to modularize bottom-up before I can benefit from modules? If my dependency does add 'module foo' to their header files, or whatever the current syntax is, can I continue to #include <foo> or is that a source incompatible change?

I raised some of these issues a few years ago regarding the clang implementation with files named exactly module.modulemap:

http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946.html

http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946i20.html

Interestingly, GCC is taking a directory-centric approach in the driver (-fmodule-path=<dir>) as opposed to the 'add a file to your compile line for each import' that Clang and MSVC are taking:

 http://gcc.gnu.org/wiki/cxx-modules

Why is Clang not doing a directory-centric driver-interface? It seems to obviously solve problems. I wonder if modules can be a success without coordination between major compiler and buildsystem developers. That's why I made the git repo - to help work on something more concrete to see how things scale.

Having just read all of my old posts again, I still worry things like this will hinder modules 'too much' to be successful. The more (small) barriers exist, the less chance of success. If modules aren't successful, then they'll become a poisoned chalice and no one will be able to work on fixing them. That's actually exactly what I expect to happen, but I also still hope I'm just missing something :). I really want to see a committee document from the people working on modules which actually explores the problems and barriers to adoption and concludes with 'none of those things matter'. I think it's fixable, but I haven't seen anyone interested enough to fix the problems (or even to find out what they are).

Anyway, you are not here for my rants.

My current very simplistic prototype, to build a module file, its respective module object file, and include those in the library/link for anything that depends on this library:

  add_custom_command(
          COMMAND ${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS} -xc++ -c -Xclang -emit-module -fmodules -fmodule-name=Hello ${CMAKE_CURRENT_SOURCE_DIR}/module.modulemap -o ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm -Xclang -fmodules-codegen
          DEPENDS module.modulemap hello.h

Why does this command depend on hello.h? If that is changed and module.modulemap is not, what will happen?

          OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm
          COMMENT "Generating hello_module.pcm"
  )
  add_library (Hello hello.cxx ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm)
  target_include_directories(Hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
  target_compile_options(Hello PUBLIC -fmodules -Xclang -fmodule-file=${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm)

(this is based on the example in the CMake docs using Hello/Demo)

Good that you got something working.

This also required one modification to CMake itself to classify a pcm file as a C++ file that needs to be compiled (to execute the 3rd line in the basic command list shown above).

An alternative to patching CMake might be

 set_source_files_properties(${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm PROPERTIES LANGUAGE CXX)

though hopefully that is also temporary.

But this isn't ideal - I don't /think/ I've got the dependencies quite right & things might not be rebuilding at the right times.
Also it involves hardcoding a bunch of things like the pcm file names, header files, etc.

Indeed. I think part of that comes from the way modules have been designed. The TS has similar issues.

Ideally, at least for a simplistic build, I wouldn't mind generating a modulemap from all the .h files (& have those headers listed in the add_library command - perhaps splitting public and private headers in some way, only including the public headers in the module file, likely). Eventually for the standards-proposal version, it's expected that there won't be any modulemap file, but maybe all headers are included in the module compilation (just pass the headers directly to the compiler).

In a design based on passing directories instead of files, would those directories be redundant with the include directories?

One of the problems modules adoption will hit is that all the compilers are designing fundamentally different command line interfaces for them. Buildsystems will have to be rewritten to take advantage of modules, and they will be annoying to use and adopt.

This also doesn't start to approach the issue of how to build modules for external libraries - which I'm happy to discuss/prototype too, though interested in working to streamline the inter-library but intra-project (not inter-project) case first.

Yes, there are many aspects to consider.

Are you interested in design of a CMake abstraction for this stuff? I have thoughts on that, but I don't know if your level of interest stretches that far.

Stephen - I saw you were asking some questions about this here ( https://groups.google.com/a/isocpp.org/forum/#!topic/modules/sDIYoU8Uljw & https://github.com/steveire/ModulesExperiments - didn't really understand how this example applied/worked, though - I guess maybe it's a prototype syntax proposal?)

It is a set of pre-modules libraries, some of which depend on one another and with some transitive dependencies in the headers.

I made it to be 'a few steps above trivial' in the hope that someone would show me how to port it to modules-ts (even if the result does not build). So far, no one has.

Can you help? It would really help my understanding of where things currently stand with modules. For example, is there only one way to port the contents of the cpp files?

After that, is there one importable module per class or one per shared library (which I think would make more sense for Qt)? And is transitive dependency expressed in the header files after porting? I think that last one is dealt with by the 'export import' syntax

The git repo is an attempt to make the discussion concrete because it would show how multiple classes and multiple libraries with dependencies could interact in a modules world. I'm interested in what it would look like ported to modules-ts, because as far as I know, clang-modules and module maps would not need porting of the cpp files at all.

Basically: What do folks think about supporting these sort of features in CMake C++ Builds? Any pointers on how I might best implement this with or without changes to CMake?

I think some design is needed up front. I expect CMake would want to have a first-class (on equal footing with include directories or compile definitions and with particular handling) concept for modules, extending the install(TARGET) command to install module binary files etc.

To do that kind of design, I at least would need to be able to experiment or conceptualize examples which are not totally trivial, such as the starting point in my repo.

On the CMake side, I think something like this should be the target (note that no compiler command line interface works like this today, which I think is a barrier to adoption):

 add_library(foo foo.cpp)

 # Target property to enable modules for the target
 set_property(TARGET foo PROPERTY USE_CXX_MODULES ON)

 # Note: Use target_include_directories to specify module search
 # paths (how GCC and MSVC work)
 # Also note: compilers should use the -I paths as a module path search list so
 # that CMake does not have to pass the same list as both -I and as -fmodule-path=
 # or similar entries.
 # Also note: This is source compatible with the cmake code that already exists!
 # The existance of /opt/bar/bing.<ext> makes 'import bing;' work.
 target_include_directories(foo PRIVATE /opt/bar)

 # Possibly need a new command to specify headers (there is
 # other motivation for this in CMake, so use a generic name without 'modules' in it)
 # Because foo has USE_CXX_MODULES ON, foo.h is processed as a module
 # and a binary representation is created for import. Other properties can
 # be set on foo.h with set_source_files_properties() to pass other command line
 # options when generating the module.
 target_headers(foo PRIVATE foo.h)

Also - in the right design, CMake does not have to regenerate -fmodule-file or whatever into the compile line any time the user adds 'import something;', which is the case with clang now afaik. Please correct me if that is not correct.

I know some people at Kitware have been thinking about modules though, so I'd be interested in any other insights from there. Brad, can you comment?

Here's some other reading material for anyone else following along:

https://izzys.casa/posts/millennials-are-killing-the-modules-ts.html
https://build2.org/article/cxx-modules-misconceptions.xhtml


Thanks,

Stephen.


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Experiments in CMake support for Clang (header & standard) modules

Stephen Kelly-2
On 04/30/2018 11:30 PM, Stephen Kelly wrote:
> Interestingly, GCC is taking a directory-centric approach in the
> driver (-fmodule-path=<dir>) as opposed to the 'add a file to your
> compile line for each import' that Clang and MSVC are taking:
>
>  http://gcc.gnu.org/wiki/cxx-modules
>
> Why is Clang not doing a directory-centric driver-interface? It seems
> to obviously solve problems.

I just discovered `-fprebuilt-module-path=`. I'm glad it exists (is it
'new'?), but why don't you just use the -I paths?

Thanks,

Steve.

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Experiments in CMake support for Clang (header & standard) modules

David Blaikie
In reply to this post by Stephen Kelly-2
On Mon, Apr 30, 2018 at 3:30 PM Stephen Kelly <[hidden email]> wrote:
On 04/20/2018 01:39 AM, David Blaikie wrote:
Hi there,

I'm experimenting with creating examples (& potential changes to CMake itself, if needed/useful) of building clang modules (currently using the semi-backwards compatible "header modules", with the intent of also/moving towards supporting pre-standard C++ modules in development in Clang).

Great! Thanks for reaching out. Sorry it has taken me a while to respond.

No worries - thanks for getting back to me!
 
Have you had other response off-list?

Nah - chatted a little with coworkers (fellow Clang/LLVM developers - mostly Richard Smith & Chandler Carruth) off-list, but nothing much more.
The basic commands required are:

  clang++ -fmodules -xc++ -Xclang -emit-module -Xclang -fmodules-codegen -fmodule-name=foo foo.modulemap -o foo.pcm
  clang++ -fmodules -c -fmodule-file=foo.pcm use.cpp
  clang++ -c foo.pcm
  clang++ foo.o use.o -o a.out

Ok. Fundamentally, I am suspicious of having to have a -fmodule-file=foo.pcm for every 'import foo' in each cpp file. I shouldn't have to manually add that each time I add a new import to my cpp file. Even if it can be automated (eg by CMake), I shouldn't have to have my buildsystem be regenerated each time I add an import to my cpp file either.

That's something I mentioned in the google groups post I made which you linked to. How will that work when using Qt or any other library?

- My understanding/feeling is that this would be similar to how a user has to change their link command when they pick up a new dependency.

Nope, scratch that ^ I had thought that was the case, but talking more with Richard Smith it seems there's an expectation that modules will be somewhere between header and library granularity (obviously some small libraries today have one or only a few headers, some (like Qt) have many - maybe those on the Qt end might have slightly fewer modules than the have headers - but still several modules to one library most likely, by the sounds of it)


Now, admittedly, external dependencies are a little more complicated than internal (within a single project consisting of multiple libraries) - which is why I'd like to focus a bit on the simpler internal case first.
 
Today, a beginner can find a random C++ book, type in a code example from chapter one and put `g++ -I/opt/book_examples prog1.cpp` into a terminal and get something compiling and running. With modules, they'll potentially have to pass a whole list of module files too.

Yeah, there's some talk of supporting a mode that doesn't explicitly build/use modules in the filesystem, but only in memory for the purpose of preserving the isolation semantics of modules. This would be used in simple direct-compilation cases like this. Such a library might need a configuration file or similar the compiler can parse to discover the parameters (warning flags, define flags, whatever else) needed to build the BMI.
 
Lots of people manually maintain Makefile-based buildsystems today, and most other companies I've been inside of have their own custom tool or bunch of python scripts, or both. Manually changing such buildsystems to add -fmodule-file or -fmodule-map-file each time an import is added is a significant barrier.

Will my project have to compile the modules files for all of my dependencies?

Yes - and that's where the external dependencies get complicated.
 
Even more complication for my buildsystem. Do I have to wait for my dependencies to modularize bottom-up before I can benefit from modules?

There are some ideas about how to handle that ('legacy headers'/modules in Richard's work/proposed amendment to the TS), but I'm trying to focus on a few of the simpler cases first.
 
If my dependency does add 'module foo' to their header files, or whatever the current syntax is, can I continue to #include <foo> or is that a source incompatible change?

I believe it'd be a source incompatible change. You could continue to provide a header for your module separately. They'd likely have different extensions anyway.
 
I raised some of these issues a few years ago regarding the clang implementation with files named exactly module.modulemap:

http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946.html

http://clang-developers.42468.n3.nabble.com/How-do-I-try-out-C-modules-with-clang-td4041946i20.html

Interestingly, GCC is taking a directory-centric approach in the driver (-fmodule-path=<dir>) as opposed to the 'add a file to your compile line for each import' that Clang and MSVC are taking:

 http://gcc.gnu.org/wiki/cxx-modules

Why is Clang not doing a directory-centric driver-interface? It seems to obviously solve problems. I wonder if modules can be a success without coordination between major compiler and buildsystem developers. That's why I made the git repo - to help work on something more concrete to see how things scale.

'We' (myself & other Clang developers) are/will be talking to GCC folks to try to get consistency here, in one direction or another (maybe some 3rd direction different from Clang or LLVM's). As you noted in a follow-up, there is a directory-based flag in Clang now, added by Boris as he's been working through adding modules support to Build2.
 
Having just read all of my old posts again, I still worry things like this will hinder modules 'too much' to be successful. The more (small) barriers exist, the less chance of success. If modules aren't successful, then they'll become a poisoned chalice and no one will be able to work on fixing them. That's actually exactly what I expect to happen, but I also still hope I'm just missing something :). I really want to see a committee document from the people working on modules which actually explores the problems and barriers to adoption and concludes with 'none of those things matter'. I think it's fixable, but I haven't seen anyone interested enough to fix the problems (or even to find out what they are).

Indeed - hence my desire to talk through these things, get some practical experience, document them to the committee in perhaps a less-ranty, more concrete form along with pros/cons/unknowns/etc to hopefully find some consistency, maybe write up a document of "this is how we expect build systems to integrate with this C++ feature", etc.
 

Anyway, you are not here for my rants.


My current very simplistic prototype, to build a module file, its respective module object file, and include those in the library/link for anything that depends on this library:

  add_custom_command(
          COMMAND ${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS} -xc++ -c -Xclang -emit-module -fmodules -fmodule-name=Hello ${CMAKE_CURRENT_SOURCE_DIR}/module.modulemap -o ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm -Xclang -fmodules-codegen
          DEPENDS module.modulemap hello.h

Why does this command depend on hello.h?

Because it builds the binary module interface (hello_module.pcm) that is a serialized form of the compiler's internal representation of the contents of module.modulemap which refers to hello.h (the modulemap lists the header files that are part of the module). This is all using Clang's current backwards semi-compatible "header modules" stuff. In a "real" modules system, ideally there wouldn't be any modulemap. Just a .cppm file, and any files it depends on (discovered through the build system scanning the module imports, or a compiler-driven .d file style thing).

Perhaps it'd be better for me to demonstrate something closer to the actual modules reality, rather than this retro header modules stuff that clang supports.
 
If that is changed and module.modulemap is not, what will happen?

If hello.h is changed and module.modulemap is not changed? The hello_module.pcm does need to be rebuilt.

Ideally all of this would be implicit (maybe with some flag/configuration, or detected based on new file extensions for C++ interface definitions) in the add_library - taking, let's imagine, the .ccm (let's say, for argument's sake*) file listed in the add_library's inputs and using it to build a .pcm (BMI), building that .pcm as an object file along with all the normal .cc files, 


* alternatively, maybe they'll all just be .cc files & a build system would be scanning the .cc files to figure out dependencies & could notice that one of them is the blessed module interface definition based on the first line in the file.
 


          OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm
          COMMENT "Generating hello_module.pcm"
  )
  add_library (Hello hello.cxx ${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm)
  target_include_directories(Hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
  target_compile_options(Hello PUBLIC -fmodules -Xclang -fmodule-file=${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm)

(this is based on the example in the CMake docs using Hello/Demo)

Good that you got something working.


This also required one modification to CMake itself to classify a pcm file as a C++ file that needs to be compiled (to execute the 3rd line in the basic command list shown above).

An alternative to patching CMake might be

 set_source_files_properties(${CMAKE_CURRENT_BINARY_DIR}/hello_module.pcm PROPERTIES LANGUAGE CXX)

though hopefully that is also temporary.

Ah, thanks! I'll give that a go.

So I suppose the more advanced question: Is there a way I can extend handling of existing CXX files (and/or define a new kind of file, say, CXXM?) specified in a cc_library. If I want to potentially check if a .cc file is a module, discover its module dependencies, add new rules about how to build those, etc. Is that do-able within my cmake project, or would that require changes to cmake itself? (I'm happy to poke around at what those changes might look like)
 


But this isn't ideal - I don't /think/ I've got the dependencies quite right & things might not be rebuilding at the right times.
Also it involves hardcoding a bunch of things like the pcm file names, header files, etc.

Indeed. I think part of that comes from the way modules have been designed. The TS has similar issues.

Sure - but I'd still be curious to understand how I might go about modifying the build system to handle this. If there are obvious things I have gotten wrong about the dependencies, etc, that would cause this not to rebuild on modifications to any of the source/header files - I'd love any tips you've got.

& if there are good paths forward for ways to prototype changes to the build system to handle, say, specifying a switch/setting a property/turning on a feature that I could implement that would collect all the .ccm files in an add_library rule and use them to make a .pcm file - I'd be happy to try prototyping that.
Ideally, at least for a simplistic build, I wouldn't mind generating a modulemap from all the .h files (& have those headers listed in the add_library command - perhaps splitting public and private headers in some way, only including the public headers in the module file, likely). Eventually for the standards-proposal version, it's expected that there won't be any modulemap file, but maybe all headers are included in the module compilation (just pass the headers directly to the compiler).

In a design based on passing directories instead of files, would those directories be redundant with the include directories?

I'm not sure I understand the question, but if I do, I think the answer would be: no, they wouldn't be redundant. The system will not have precompiled modules available to use - because binary module definitions are compiler (& compiler version, and to some degree, compiler flags (eg: are you building this for x86 32 bit or 64 bit?)) dependent.
 
One of the problems modules adoption will hit is that all the compilers are designing fundamentally different command line interfaces for them.

*nod* We'll be working amongst GCC and Clang at least to try to converge on something common.
 
Buildsystems will have to be rewritten to take advantage of modules, and they will be annoying to use and adopt.

I'm hoping to minimize any rewriting - but also potentially provide recipes for CMake (& other users) as well as patches to CMake itself to help facilitate this.
This also doesn't start to approach the issue of how to build modules for external libraries - which I'm happy to discuss/prototype too, though interested in working to streamline the inter-library but intra-project (not inter-project) case first.

Yes, there are many aspects to consider.

Are you interested in design of a CMake abstraction for this stuff? I have thoughts on that, but I don't know if your level of interest stretches that far.

Not sure how much work it'd be - at the moment my immediate interest is to show as much real-world/can-actually-run prototype with cmake as possible, either with or without changes to cmake itself (or a combination of minimal cmake changes plus project-specific recipes of how to write a user's cmake files to work with this stuff) or also showing non-working/hypothetical prototypes of what ideal user cmake files would look like with reasonable/viable (but not yet implemented) cmake support.
Stephen - I saw you were asking some questions about this here ( https://groups.google.com/a/isocpp.org/forum/#!topic/modules/sDIYoU8Uljw & https://github.com/steveire/ModulesExperiments - didn't really understand how this example applied/worked, though - I guess maybe it's a prototype syntax proposal?)

It is a set of pre-modules libraries, some of which depend on one another and with some transitive dependencies in the headers.

I made it to be 'a few steps above trivial' in the hope that someone would show me how to port it to modules-ts (even if the result does not build). So far, no one has.

Ah, OK.
 
Can you help? It would really help my understanding of where things currently stand with modules.

I can certainly have a go, for sure.
 
For example, is there only one way to port the contents of the cpp files?

Much like header grouping - how granular headers are (how many headers you have for a given library) is up to the developer to some degree (certain things can't be split up), similarly with modules - given a set of C++ definitions, it's not 100% constrained how those definitions are exposed as modules - the developer has some freedom over how the declarations of those entities are grouped into modules.
 
After that, is there one importable module per class or one per shared library (which I think would make more sense for Qt)?

Apparently (this was a surprise to me - since I'd been thinking about this based on the Clang header modules (backwards compatibility stuff, not the standardized/new language feature modules)) the thinking is probably somewhere between one-per-class and one-per-shared-library. But for me, in terms of how a build file would interact with this, more than one-per-shared-library is probably the critical tipping point. If it was just one per shared library, then I'd feel like the dependency/flag management would be relatively simple. You have to add a flag to the linker commandline to link in a library, so you have to add a flag to the compile step to reference a module, great. But, no, bit more complicated than that given the finer granularity that's expected here.
 
And is transitive dependency expressed in the header files after porting? I think that last one is dealt with by the 'export import' syntax

If you mean one module exposing things from modules it depends on - yes, you can export import. (but by default your imports are just accessible to the implementation of the module)
 
The git repo is an attempt to make the discussion concrete because it would show how multiple classes and multiple libraries with dependencies could interact in a modules world. I'm interested in what it would look like ported to modules-ts, because as far as I know, clang-modules and module maps would not need porting of the cpp files at all.

Right, clang header-modules is a backwards compatibility feature. It does require a constrained subset of C++ to be used to be effective (ie: basically your headers need to be what we think of as ideal/canonical headers - reincludable, independent, complete, etc). So if you've got good/isolated headers, you can port them to Clang's header modules by adding the module maps & potentially not doing anything else - though, if you rely on not changing your build system, then that presents some problems if you want to scale (more cores) or distribute your build. Because the build system doesn't know about these  dependencies - so if you have, say, two .cc files that both include foo.h then bar.h - well, the build system runs two compiles, both compiles try to implicitly build the foo.h module - one blocks waiting for the other to complete, then they continue and block again waiting for bar.h module to be built. If the build system knew about these dependencies (what Google uses - what we call "explicit (header)modules") then it could build the foo.h module and the bar.h module in parallel, then build the two .cc files in parallel.
 


Basically: What do folks think about supporting these sort of features in CMake C++ Builds? Any pointers on how I might best implement this with or without changes to CMake?

I think some design is needed up front. I expect CMake would want to have a first-class (on equal footing with include directories or compile definitions and with particular handling) concept for modules, extending the install(TARGET) command to install module binary files etc.

Module binary files wouldn't be installed in the sense of being part of the shipped package of a library - because module binary files are compiler/flag/etc specific.
 
To do that kind of design, I at least would need to be able to experiment or conceptualize examples which are not totally trivial, such as the starting point in my repo.

On the CMake side, I think something like this should be the target (note that no compiler command line interface works like this today, which I think is a barrier to adoption):

 add_library(foo foo.cpp)

 # Target property to enable modules for the target
 set_property(TARGET foo PROPERTY USE_CXX_MODULES ON)

 # Note: Use target_include_directories to specify module search
 # paths (how GCC and MSVC work)
 # Also note: compilers should use the -I paths as a module path search list so
 # that CMake does not have to pass the same list as both -I and as -fmodule-path=
 # or similar entries.
 # Also note: This is source compatible with the cmake code that already exists!
 # The existance of /opt/bar/bing.<ext> makes 'import bing;' work.
 target_include_directories(foo PRIVATE /opt/bar)

 # Possibly need a new command to specify headers (there is
 # other motivation for this in CMake, so use a generic name without 'modules' in it)
 # Because foo has USE_CXX_MODULES ON, foo.h is processed as a module
 # and a binary representation is created for import. Other properties can
 # be set on foo.h with set_source_files_properties() to pass other command line
 # options when generating the module.
 target_headers(foo PRIVATE foo.h)

Also - in the right design, CMake does not have to regenerate -fmodule-file or whatever into the compile line any time the user adds 'import something;', which is the case with clang now afaik. Please correct me if that is not correct.

I know some people at Kitware have been thinking about modules though, so I'd be interested in any other insights from there. Brad, can you comment?

Here's some other reading material for anyone else following along:

https://izzys.casa/posts/millennials-are-killing-the-modules-ts.html
https://build2.org/article/cxx-modules-misconceptions.xhtml

Thanks for the links & questions/ideas!

- Dave
 



Thanks,

Stephen.


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake