Project Structure and add_subdirectory()

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Project Structure and add_subdirectory()

Marek Vojtko
Using add_subdirectory() in a "superbuild" setup is straightforward: Create a root CMakeLists.txt which calls add_subdirectory() on all directories of your project and each target can then use target_link_libraries() without worrying where the target came from.

Unfortunately the "superbuild" setup also means that each target is compiled exactly once, which means that you cannot change a target's build environment per-application (e.g. preprocessor definitions that change the sizes of stack arrays).

If I need per-application control of its dependencies' build environment I can have each application's CMakeLists.txt call add_subdirectory() for all of its dependencies. However, that means that each application now has to know all of its *transitive* dependencies and the application's CMakeLists.txt has to be kept up-to-date with any changes in its dependency tree, no matter how deep. So if AppA depends on LibA, which depends on LibB, which depends on LibC. Not only does AppA's CMakeLists.txt have to call add_subdirectory() on LibA, LibB, and LibC, but if LibC is modified to depend on LibX, now AppA's CMakeLists.txt has to also be modified to call add_subdirectory() on LibX.

Having each target call add_subdirectory() on its own dependencies seems silly, because that would create an insane number of duplicated targets. If LibA depends on LibB and LibC, and both LibB and LibC depend on LibD, this approach would result in both LibB and LibC calling add_subdirectory() on LibD, creating the target twice, which would likely not compile.

Are these my only two options? Either:
- use a superbuild to have CMake handle my transitive dependencies, but give up per-application build environment changes, or
- track all transitive dependencies manually in each application's CMakeLists.txt, but retain the ability to change the build environment per application.
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Project Structure and add_subdirectory()

Avraham Shukron
I think option 2 (keeping track on transitive dependencies) should be out of the question.
It is bound to fail.

Superbuild setup makes sense where all the components are part of a bigger whole.
But if each application may depend on a different "flavor" of  a dependency library, I think it should be reflected in your project structure, E.g each application in a separate repository, with all the dependencies as submodules.

alternatively, you can create a multi-root setup, where you have a project-root-directory for each application, with a top-level CMakeLists that adds all the necessary subdirectories.
This way, you get the best of all worlds:
1. Each component's source code is kept only once, and changes are immediately affecting all products.
2. You can compile each application independently from one another.

Consider the following structure:

-top/
    - app1_root/
        - CMakeLists.txt
    - app2_root/
        - CMakeLists.txt
    - app3_root/
        - CMakeLists.txt
    - components/
        - CMakeLists.txt
        - libA/
        - libB/
        - libC/

The only issue I can think of is that in each application's top-level CMakeLists.txt you have to add_subdirectory(../components) and the ".." is kind of ugly.
Other than that I think it should work


On Fri, Sep 27, 2019 at 7:00 PM Marek Vojtko <[hidden email]> wrote:
Using add_subdirectory() in a "superbuild" setup is straightforward: Create a root CMakeLists.txt which calls add_subdirectory() on all directories of your project and each target can then use target_link_libraries() without worrying where the target came from.

Unfortunately the "superbuild" setup also means that each target is compiled exactly once, which means that you cannot change a target's build environment per-application (e.g. preprocessor definitions that change the sizes of stack arrays).

If I need per-application control of its dependencies' build environment I can have each application's CMakeLists.txt call add_subdirectory() for all of its dependencies. However, that means that each application now has to know all of its *transitive* dependencies and the application's CMakeLists.txt has to be kept up-to-date with any changes in its dependency tree, no matter how deep. So if AppA depends on LibA, which depends on LibB, which depends on LibC. Not only does AppA's CMakeLists.txt have to call add_subdirectory() on LibA, LibB, and LibC, but if LibC is modified to depend on LibX, now AppA's CMakeLists.txt has to also be modified to call add_subdirectory() on LibX.

Having each target call add_subdirectory() on its own dependencies seems silly, because that would create an insane number of duplicated targets. If LibA depends on LibB and LibC, and both LibB and LibC depend on LibD, this approach would result in both LibB and LibC calling add_subdirectory() on LibD, creating the target twice, which would likely not compile.

Are these my only two options? Either:
- use a superbuild to have CMake handle my transitive dependencies, but give up per-application build environment changes, or
- track all transitive dependencies manually in each application's CMakeLists.txt, but retain the ability to change the build environment per application.
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Project Structure and add_subdirectory()

Marek Vojtko

I agree that manually keeping track of transitive dependencies is a very bad and error-prone idea.

 

The ideal solution is the separate repositories, but I don’t think that’s really feasible.

 

The multi-root setup would work, but the ugly part of it is requiring an “app*_root” directory for each application, which is needed to be able to create a solution/makefile with only the application’s dependencies. When compiling an app I don’t want to be compiling targets it does not depend on, which is what would happen if I were to use the app*_root directory directly. So in your example:

 

-top/

    - app1 /

        - CMakeLists.txt

    - app1_root/

        - CMakeLists.txt

    - app2/

        - CMakeLists.txt

    - app2_root/

        - CMakeLists.txt

    - app3/

        - CMakeLists.txt

    - app3_root/

        - CMakeLists.txt

    - components/

        - CMakeLists.txt

        - libA/

        - libB/

        - libC/

 

And developers would have to CMake the app*_root directories and then open solutions / run makefiles generated for the individual apps. Not terrible, but definitely not intuitive. But thank you for sharing that, I haven’t thought of that 😊

 

From: Avraham Shukron <[hidden email]>
Sent: Friday, September 27, 2019 5:23 PM
To: Marek Vojtko <[hidden email]>
Cc: [hidden email]
Subject: Re: [CMake] Project Structure and add_subdirectory()

 

I think option 2 (keeping track on transitive dependencies) should be out of the question.

It is bound to fail.

 

Superbuild setup makes sense where all the components are part of a bigger whole.

But if each application may depend on a different "flavor" of  a dependency library, I think it should be reflected in your project structure, E.g each application in a separate repository, with all the dependencies as submodules.

 

alternatively, you can create a multi-root setup, where you have a project-root-directory for each application, with a top-level CMakeLists that adds all the necessary subdirectories.

This way, you get the best of all worlds:

1. Each component's source code is kept only once, and changes are immediately affecting all products.

2. You can compile each application independently from one another.

 

Consider the following structure:

 

-top/

    - app1_root/

        - CMakeLists.txt

    - app2_root/

        - CMakeLists.txt

    - app3_root/

        - CMakeLists.txt

    - components/

        - CMakeLists.txt

        - libA/

        - libB/

        - libC/

 

The only issue I can think of is that in each application's top-level CMakeLists.txt you have to add_subdirectory(../components) and the ".." is kind of ugly.

Other than that I think it should work

 

 

On Fri, Sep 27, 2019 at 7:00 PM Marek Vojtko <[hidden email]> wrote:

Using add_subdirectory() in a "superbuild" setup is straightforward: Create a root CMakeLists.txt which calls add_subdirectory() on all directories of your project and each target can then use target_link_libraries() without worrying where the target came from.

Unfortunately the "superbuild" setup also means that each target is compiled exactly once, which means that you cannot change a target's build environment per-application (e.g. preprocessor definitions that change the sizes of stack arrays).

If I need per-application control of its dependencies' build environment I can have each application's CMakeLists.txt call add_subdirectory() for all of its dependencies. However, that means that each application now has to know all of its *transitive* dependencies and the application's CMakeLists.txt has to be kept up-to-date with any changes in its dependency tree, no matter how deep. So if AppA depends on LibA, which depends on LibB, which depends on LibC. Not only does AppA's CMakeLists.txt have to call add_subdirectory() on LibA, LibB, and LibC, but if LibC is modified to depend on LibX, now AppA's CMakeLists.txt has to also be modified to call add_subdirectory() on LibX.

Having each target call add_subdirectory() on its own dependencies seems silly, because that would create an insane number of duplicated targets. If LibA depends on LibB and LibC, and both LibB and LibC depend on LibD, this approach would result in both LibB and LibC calling add_subdirectory() on LibD, creating the target twice, which would likely not compile.

Are these my only two options? Either:
- use a superbuild to have CMake handle my transitive dependencies, but give up per-application build environment changes, or
- track all transitive dependencies manually in each application's CMakeLists.txt, but retain the ability to change the build environment per application.
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Project Structure and add_subdirectory()

Avraham Shukron

On Sat, Sep 28, 2019 at 1:39 AM Marek Vojtko <[hidden email]> wrote:

I agree that manually keeping track of transitive dependencies is a very bad and error-prone idea.

 

The ideal solution is the separate repositories, but I don’t think that’s really feasible.

 

The multi-root setup would work, but the ugly part of it is requiring an “app*_root” directory for each application, which is needed to be able to create a solution/makefile with only the application’s dependencies. When compiling an app I don’t want to be compiling targets it does not depend on, which is what would happen if I were to use the app*_root directory directly.


Why would you build targets you don't depend upon in my original proposal? you certainly include them all in your project, but as long as you build your application target, only it's actual dependencies will be built. Of course, if you build the "all" target, everything will be built, but that could easily be avoided with EXCLUDE_FROM_ALL when adding the `components` subdirectory.😊

 

From: Avraham Shukron <[hidden email]>
Sent: Friday, September 27, 2019 5:23 PM
To: Marek Vojtko <[hidden email]>
Cc: [hidden email]
Subject: Re: [CMake] Project Structure and add_subdirectory()

 

I think option 2 (keeping track on transitive dependencies) should be out of the question.

It is bound to fail.

 

Superbuild setup makes sense where all the components are part of a bigger whole.

But if each application may depend on a different "flavor" of  a dependency library, I think it should be reflected in your project structure, E.g each application in a separate repository, with all the dependencies as submodules.

 

alternatively, you can create a multi-root setup, where you have a project-root-directory for each application, with a top-level CMakeLists that adds all the necessary subdirectories.

This way, you get the best of all worlds:

1. Each component's source code is kept only once, and changes are immediately affecting all products.

2. You can compile each application independently from one another.

 

Consider the following structure:

 

-top/

    - app1_root/

        - CMakeLists.txt

    - app2_root/

        - CMakeLists.txt

    - app3_root/

        - CMakeLists.txt

    - components/

        - CMakeLists.txt

        - libA/

        - libB/

        - libC/

 

The only issue I can think of is that in each application's top-level CMakeLists.txt you have to add_subdirectory(../components) and the ".." is kind of ugly.

Other than that I think it should work

 

 

On Fri, Sep 27, 2019 at 7:00 PM Marek Vojtko <[hidden email]> wrote:

Using add_subdirectory() in a "superbuild" setup is straightforward: Create a root CMakeLists.txt which calls add_subdirectory() on all directories of your project and each target can then use target_link_libraries() without worrying where the target came from.

Unfortunately the "superbuild" setup also means that each target is compiled exactly once, which means that you cannot change a target's build environment per-application (e.g. preprocessor definitions that change the sizes of stack arrays).

If I need per-application control of its dependencies' build environment I can have each application's CMakeLists.txt call add_subdirectory() for all of its dependencies. However, that means that each application now has to know all of its *transitive* dependencies and the application's CMakeLists.txt has to be kept up-to-date with any changes in its dependency tree, no matter how deep. So if AppA depends on LibA, which depends on LibB, which depends on LibC. Not only does AppA's CMakeLists.txt have to call add_subdirectory() on LibA, LibB, and LibC, but if LibC is modified to depend on LibX, now AppA's CMakeLists.txt has to also be modified to call add_subdirectory() on LibX.

Having each target call add_subdirectory() on its own dependencies seems silly, because that would create an insane number of duplicated targets. If LibA depends on LibB and LibC, and both LibB and LibC depend on LibD, this approach would result in both LibB and LibC calling add_subdirectory() on LibD, creating the target twice, which would likely not compile.

Are these my only two options? Either:
- use a superbuild to have CMake handle my transitive dependencies, but give up per-application build environment changes, or
- track all transitive dependencies manually in each application's CMakeLists.txt, but retain the ability to change the build environment per application.
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Project Structure and add_subdirectory()

Marek Vojtko

> Why would you build targets you don't depend upon in my original proposal? you certainly include them all in your project, but as long as you build your application target, only it's actual dependencies will be built. Of course, if you build the "all" target, everything will be built, but that could easily be avoided with EXCLUDE_FROM_ALL when adding the `components` subdirectory.😊

 

Unfortunately, we’re using Visual Studio, which makes it very easy to “Build Solution” (which builds all targets, including the CMake-provided ALL_BUILD target), but more cumbersome to build just the application target. There is a default shortcut key for “Build Solution,” but to build just the application you have to use the UI menus :(

 

Even if the EXCLUDE_FROM_ALL worked – again, I believe Visual Studio’s “Build Solution” just builds all Visual Studio projects that are part of the solution, regardless of what ALL_BUILD does – having to track which libraries should be excluded from the ALL_BUILD target for each application boils down to manually tracking transitive dependencies 😊


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake