Interfacing cmake with 3rdparties package manager, and exclude them from ninja clean

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Interfacing cmake with 3rdparties package manager, and exclude them from ninja clean

david.jobet
Hello,

I'm working on a project where some of our libs depend on custom build of some external libraries.
Those external libraries are managed through a repository manager similar to rpm (redhat package manager) which allows us to retrieve pre-compiled versions with related header files. (similar to devel packages)

The catch is we want libraries to be downloaded automatically to a shared (multiple users) local dir __AND__ we don't want them to be cleaned. (because another user could be using it).

I've tried add_custom_command but files disappear with make clean.
I've tried to set NO_CUSTOM_CLEAN, that works with make but not with ninja.

I've tried to chain add_custom_target to download the file and add_dependencies, again, this works with make but not with ninja (ninja does not know how to create protocol buffer compiler "protoc" for example since it's not the output of any command he knows)

I've tried externalproject_add with BUILD_BYPRODUCTS but it looks like ninja still clean the downloaded files.

Any idea on how to make ninja not clean those downloaded files (we use generators like protoc, libraries and header files)

With regards

David

PS : in my add_custom_command/add_custom_target, I invoke "cmake -P" on a custom cmake script with 2 args. The first arg is a file to test for existence, the second one is the parameter to our custom "rpm" to download the package.

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Interfacing cmake with 3rdparties package manager, and exclude them from ninja clean

Tamás Kenéz
Maybe executing your script in configure time (execute_process)?

On Fri, Jan 20, 2017 at 4:36 PM, <[hidden email]> wrote:
Hello,

I'm working on a project where some of our libs depend on custom build of some external libraries.
Those external libraries are managed through a repository manager similar to rpm (redhat package manager) which allows us to retrieve pre-compiled versions with related header files. (similar to devel packages)

The catch is we want libraries to be downloaded automatically to a shared (multiple users) local dir __AND__ we don't want them to be cleaned. (because another user could be using it).

I've tried add_custom_command but files disappear with make clean.
I've tried to set NO_CUSTOM_CLEAN, that works with make but not with ninja.

I've tried to chain add_custom_target to download the file and add_dependencies, again, this works with make but not with ninja (ninja does not know how to create protocol buffer compiler "protoc" for example since it's not the output of any command he knows)

I've tried externalproject_add with BUILD_BYPRODUCTS but it looks like ninja still clean the downloaded files.

Any idea on how to make ninja not clean those downloaded files (we use generators like protoc, libraries and header files)

With regards

David

PS : in my add_custom_command/add_custom_target, I invoke "cmake -P" on a custom cmake script with 2 args. The first arg is a file to test for existence, the second one is the parameter to our custom "rpm" to download the package.

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Interfacing cmake with 3rdparties package manager, and exclude them from ninja clean

david.jobet
Hello,

Tx for the idea !
That would work to "boostrap" the build but I'm not very fond of it due to it's "static" nature : it would run only once during the initial cmake run.

We extract versionning information from git tags : in our current (slow) build system, a git update can trigger a "rpm" update with a newer version, so it has to be more "dynamic".

I was thinking maybe of creating links in our build tree (they would be the output of a add_custom_command) so it's ok to clean them. Creating them would involve installing the package then creating the link from the build dir to the "rpm" repo.

I think that would work but I hope there is a simpler solution.

Is it possible for external projects not to be cleaned for example ?

With regards,

David


Le 20 janv. 2017 22:13, Tamás Kenéz <[hidden email]> a écrit :
Maybe executing your script in configure time (execute_process)?

On Fri, Jan 20, 2017 at 4:36 PM, <[hidden email]> wrote:
Hello,

I'm working on a project where some of our libs depend on custom build of some external libraries.
Those external libraries are managed through a repository manager similar to rpm (redhat package manager) which allows us to retrieve pre-compiled versions with related header files. (similar to devel packages)

The catch is we want libraries to be downloaded automatically to a shared (multiple users) local dir __AND__ we don't want them to be cleaned. (because another user could be using it).

I've tried add_custom_command but files disappear with make clean.
I've tried to set NO_CUSTOM_CLEAN, that works with make but not with ninja.

I've tried to chain add_custom_target to download the file and add_dependencies, again, this works with make but not with ninja (ninja does not know how to create protocol buffer compiler "protoc" for example since it's not the output of any command he knows)

I've tried externalproject_add with BUILD_BYPRODUCTS but it looks like ninja still clean the downloaded files.

Any idea on how to make ninja not clean those downloaded files (we use generators like protoc, libraries and header files)

With regards

David

PS : in my add_custom_command/add_custom_target, I invoke "cmake -P" on a custom cmake script with 2 args. The first arg is a file to test for existence, the second one is the parameter to our custom "rpm" to download the package.

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re : Re: Interfacing cmake with 3rdparties package manager, and exclude them from ninja clean

david.jobet
In reply to this post by Tamás Kenéz
Hello,

I actually gave it a try : it works fine so long versions are hardcoded somewhere in the CMakeLists.txt and it make things really simpler.

But it does not work when version is extracted from git repository.
I came to realize I can extract this version number at build time (using add_custom_target and cmake -P), and I can even include a generated cmake file that contains the version number, but I don't know how to tell cmake to trigger a rebuild_cache anyway.
So I don't know how to make that behaviour "dynamic".
Any idea ?

With regards

David

----- Tamás Kenéz <[hidden email]> a écrit :

> Maybe executing your script in configure time (execute_process)?
>
> On Fri, Jan 20, 2017 at 4:36 PM, <[hidden email]> wrote:
>
> > Hello,
> >
> > I'm working on a project where some of our libs depend on custom build of
> > some external libraries.
> > Those external libraries are managed through a repository manager similar
> > to rpm (redhat package manager) which allows us to retrieve pre-compiled
> > versions with related header files. (similar to devel packages)
> >
> > The catch is we want libraries to be downloaded automatically to a shared
> > (multiple users) local dir __AND__ we don't want them to be cleaned.
> > (because another user could be using it).
> >
> > I've tried add_custom_command but files disappear with make clean.
> > I've tried to set NO_CUSTOM_CLEAN, that works with make but not with ninja.
> >
> > I've tried to chain add_custom_target to download the file and
> > add_dependencies, again, this works with make but not with ninja (ninja
> > does not know how to create protocol buffer compiler "protoc" for example
> > since it's not the output of any command he knows)
> >
> > I've tried externalproject_add with BUILD_BYPRODUCTS but it looks like
> > ninja still clean the downloaded files.
> >
> > Any idea on how to make ninja not clean those downloaded files (we use
> > generators like protoc, libraries and header files)
> >
> > With regards
> >
> > David
> >
> > PS : in my add_custom_command/add_custom_target, I invoke "cmake -P" on a
> > custom cmake script with 2 args. The first arg is a file to test for
> > existence, the second one is the parameter to our custom "rpm" to download
> > the package.
> >
> > --
> >
> > Powered by www.kitware.com
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> > http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Kitware offers various services to support the CMake community. For more
> > information on each offering, please visit:
> >
> > CMake Support: http://cmake.org/cmake/help/support.html
> > CMake Consulting: http://cmake.org/cmake/help/consulting.html
> > CMake Training Courses: http://cmake.org/cmake/help/training.html
> >
> > Visit other Kitware open-source projects at http://www.kitware.com/
> > opensource/opensource.html
> >
> > Follow this link to subscribe/unsubscribe:
> > http://public.kitware.com/mailman/listinfo/cmake
> >

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Loading...