Build only what you need in third party libs

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Build only what you need in third party libs

Brian Davis-2

I have used boost jam before and there was a mechanism to build only what you need by specifying dependencies within the boost libraries.  bjam would then run off calculate deps and build only what I needed. In CMake is there such a feature to specify a third party library and specify only the targets you need to build as dependencies of my build target?

i.e. can I use

add_subdirectory( lib/3rdParty/Win32/boost-cmake-1_41_0 ./boost )

then create a target that only uses say boost_filesystem and only have that built? Or do I need to build the world to get a handful of needed dll's that are actually used in my project?

--
Brian J. Davis


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

Michael Jackson
I _think_ you _might_ be able to set the BUILD_PROJECTS to
"file_system;system" then do the "add_subdirectory()".
  Give it a shot and see what happens.

_________________________________________________________
Mike Jackson                  [hidden email]
BlueQuartz Software                    www.bluequartz.net
Principal Software Engineer                  Dayton, Ohio



On Sat, Dec 5, 2009 at 4:15 PM, Brian Davis <[hidden email]> wrote:

>
> I have used boost jam before and there was a mechanism to build only what
> you need by specifying dependencies within the boost libraries.  bjam would
> then run off calculate deps and build only what I needed. In CMake is there
> such a feature to specify a third party library and specify only the targets
> you need to build as dependencies of my build target?
>
> i.e. can I use
>
> add_subdirectory( lib/3rdParty/Win32/boost-cmake-1_41_0 ./boost )
>
> then create a target that only uses say boost_filesystem and only have that
> built? Or do I need to build the world to get a handful of needed dll's that
> are actually used in my project?
>
> --
> Brian J. Davis
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

Brian Davis-2
Yes this looks like an option.  Thanks for the lead.  It is not quite what I was expecting. 

This seems specific to Boost Libraries.  Which brings up 2 questions:

Is there a generic way do this for any third party source tree?
Is there going to be CMAKE variable name resolution clash potential without namespace resolution within CMake (or is there already this problem or have I missed something)?

What I would like to do is set (please bear with me as I am a complete nube to CMake)

BUILD_PROJECTS=none

Then do

add_subdirectory(boost_source_dir)

project( my_exe )
add_executable( my_exe mysource.cpp )
add_library( boost_filesystem   SHARED)
#An IMPORTED library target references a library file located outside the project. No rules are generated to build it. <- Too bad about this last statement

or

add_dependencies( my_exe boost_filesystem )


Is there any mechanism for project level resolution of third party source targets such as in bjam boost//boost_filesystem

project
    : requirements <library>../build//boost_filesystem
      <library>/boost/system//boost_system
      <hardcode-dll-paths>true
    ;


project boost/filesystem
    : source-location ../src
    : usage-requirements # pass these requirement to dependents (i.e. users)
      <link>shared:<define>BOOST_FILESYSTEM_DYN_LINK=1
      <link>static:<define>BOOST_FILESYSTEM_STATIC_LINK=1
    ;

And boost_filesystem's use:

project
    : requirements
      <library>/boost/filesystem//boost_filesystem
      <toolset>msvc:<asynch-exceptions>on
    ;

where some syntax like:

add_subdirectory(boost_source_dir PROJECT boost )

add_dependencies( my_exe boost//boost_filesystem )

or

add_library( boost//boost_filesystem   SHARED)


allowing the specification and namespace resolution of the 3rdParty (boost) library

I know above does not exist as my example question shows, but if it does in some other form what is it?

Brian

On Sat, Dec 5, 2009 at 3:45 PM, Mike Jackson <[hidden email]> wrote:
I _think_ you _might_ be able to set the BUILD_PROJECTS to
"file_system;system" then do the "add_subdirectory()".
 Give it a shot and see what happens.

_________________________________________________________
Mike Jackson                  [hidden email]
BlueQuartz Software                    www.bluequartz.net
Principal Software Engineer                  Dayton, Ohio



On Sat, Dec 5, 2009 at 4:15 PM, Brian Davis <[hidden email]> wrote:
>
> I have used boost jam before and there was a mechanism to build only what
> you need by specifying dependencies within the boost libraries.  bjam would
> then run off calculate deps and build only what I needed. In CMake is there
> such a feature to specify a third party library and specify only the targets
> you need to build as dependencies of my build target?
>
> i.e. can I use
>
> add_subdirectory( lib/3rdParty/Win32/boost-cmake-1_41_0 ./boost )
>
> then create a target that only uses say boost_filesystem and only have that
> built? Or do I need to build the world to get a handful of needed dll's that
> are actually used in my project?
>
> --
> Brian J. Davis
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
>



--
Brian J. Davis


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

Michael Jackson
Please keep on list so that others may benefit/help

So you are wanting to include the Boost sources in your project and  
you just want to build a specific subset of Boost to use with your  
project?

  I guess I might try setting the
# Set what boost libraries will be built
BUILD_PROJECTS=file_system;system

# Configure the Boost CMake distribution to build
add_subdirectory(boost-cmake0 ${PROJECT_BINARY_DIR}/Boost-Cmake0)

# look at <http://sodium.resophonic.com/boost-cmake/current-docs/exported_targets.html 
 > for
# more details about the included file. Assuming it is already  
generated by the time cmake
# hits this point.
include (  ${PROJECT_BINARY_DIR}/Boost-Cmake0/lib/Exports.cmake )

# Create your executable
add_executable(my_exe ${my_exe_sources} )
# Have your executable link to the boost-file_system library (and  
probably the boost-system library)
# Note that the 'boost_file_system' is the actual name of a target  
from the Boost-cmake0 sub project.
#  See the web page linked above for more details
target_link_libraries ( my_exe boost_file_system)

I have not tried this at all, but this would be my first attempt. The  
'target_link_libraries()' command will take care of all the  
dependencies for 'my_exe', thus making sure boost-file_system is built  
first before building the 'my_exe' executable.

Maybe someone else can comment..

_________________________________________________________
Mike Jackson                  [hidden email]
BlueQuartz Software                    www.bluequartz.net
Principal Software Engineer                  Dayton, Ohio

On Dec 7, 2009, at 11:47 AM, Brian Davis wrote:

> Yes this looks like an option.  Thanks for the lead.  It is not  
> quite what I was expecting.
>
> This seems specific to Boost Libraries.  Which brings up 2 questions:
>
> Is there a generic way do this for any third party source tree?
> Is there going to be CMAKE variable name resolution clash potential  
> without namespace resolution within CMake (or is there already this  
> problem or have I missed something)?
>
> What I would like to do is set (please bear with me as I am a  
> complete nube to CMake)
>
> BUILD_PROJECTS=none
>
> Then do
>
> add_subdirectory(boost_source_dir)
>
> project( my_exe )
> add_executable( my_exe mysource.cpp )
> add_library( boost_filesystem   SHARED)
> #An IMPORTED library target references a library file located  
> outside the project. No rules are generated to build it. <- Too bad  
> about this last statement
>
> or
>
> add_dependencies( my_exe boost_filesystem )
>
>
> Is there any mechanism for project level resolution of third party  
> source targets such as in bjam boost//boost_filesystem
>
> project
>     : requirements <library>../build//boost_filesystem
>       <library>/boost/system//boost_system
>       <hardcode-dll-paths>true
>     ;
>
>
> project boost/filesystem
>     : source-location ../src
>     : usage-requirements # pass these requirement to dependents  
> (i.e. users)
>       <link>shared:<define>BOOST_FILESYSTEM_DYN_LINK=1
>       <link>static:<define>BOOST_FILESYSTEM_STATIC_LINK=1
>     ;
>
> And boost_filesystem's use:
>
> project
>     : requirements
>       <library>/boost/filesystem//boost_filesystem
>       <toolset>msvc:<asynch-exceptions>on
>     ;
>
> where some syntax like:
>
> add_subdirectory(boost_source_dir PROJECT boost )
>
> add_dependencies( my_exe boost//boost_filesystem )
>
> or
>
> add_library( boost//boost_filesystem   SHARED)
>
>
> allowing the specification and namespace resolution of the 3rdParty  
> (boost) library
>
> I know above does not exist as my example question shows, but if it  
> does in some other form what is it?
>
> Brian
>
> On Sat, Dec 5, 2009 at 3:45 PM, Mike Jackson <[hidden email]
> > wrote:
> I _think_ you _might_ be able to set the BUILD_PROJECTS to
> "file_system;system" then do the "add_subdirectory()".
>  Give it a shot and see what happens.
>
> _________________________________________________________
> Mike Jackson                  [hidden email]
> BlueQuartz Software                    www.bluequartz.net
> Principal Software Engineer                  Dayton, Ohio
>
>
>
> On Sat, Dec 5, 2009 at 4:15 PM, Brian Davis <[hidden email]>  
> wrote:
> >
> > I have used boost jam before and there was a mechanism to build  
> only what
> > you need by specifying dependencies within the boost libraries.  
> bjam would
> > then run off calculate deps and build only what I needed. In CMake  
> is there
> > such a feature to specify a third party library and specify only  
> the targets
> > you need to build as dependencies of my build target?
> >
> > i.e. can I use
> >
> > add_subdirectory( lib/3rdParty/Win32/boost-cmake-1_41_0 ./boost )
> >
> > then create a target that only uses say boost_filesystem and only  
> have that
> > built? Or do I need to build the world to get a handful of needed  
> dll's that
> > are actually used in my project?
> >
> > --
> > Brian J. Davis
> >
> >
> > _______________________________________________
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> > http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.cmake.org/mailman/listinfo/cmake
> >
>
>
>
> --
> Brian J. Davis
>

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

troy d. straszheim
Michael Jackson wrote:
>
> So you are wanting to include the Boost sources in your project and you
> just want to build a specific subset of Boost to use with your project?
>

Here's what I came up with:

http://sodium.resophonic.com/boost-cmake/current-docs/exported_targets.html#with-boost-source-in-a-subdirectory-of-your-source

the EXCLUDE_FROM_ALL option to add_subdirectory makes things remarkably
easy... nice feature.

-t


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

Michael Jackson


On Dec 7, 2009, at 1:28 PM, troy d. straszheim wrote:

> Michael Jackson wrote:
>> So you are wanting to include the Boost sources in your project and  
>> you just want to build a specific subset of Boost to use with your  
>> project?
>
> Here's what I came up with:
>
> http://sodium.resophonic.com/boost-cmake/current-docs/exported_targets.html#with-boost-source-in-a-subdirectory-of-your-source
>
> the EXCLUDE_FROM_ALL option to add_subdirectory makes things  
> remarkably easy... nice feature.
>
> -t
>
>


Nice,
    Couple of comments though:
"Setting BUILD_PROJECTS, etc is unnecessary, but if all the extra  
boost targets bother you you can set it at the commandline as usual:"

Why can't I set BUILD_PROJECTS in my own CMakeLists.txt file? Wouldn't  
this get rid of the need to pass this on the command line? Also what  
about those running CMake-GUI? They don't have a command line to work  
from?

  By using the name of the target (boost_filesystem-mt-shared in your  
example), will CMake automatically "do the right thing" with respect  
to Debug and release libraries? Will building my project as a Debug  
pick up the Debug Boost libraries?

  I see this as having some issues? I now have to detect when someone  
wants a "Debug" build then set the "BOOST_ENABLE_DEBUG=ON" before  
calling the add_subdirectory(boost-src).

  I say all this without having tried any of this but it _is_  
intriguing me currently due to all the problems trying to get Boost  
built OS X and Windows without memory leaks, compile errors and, ya  
know, basic stuff like that. If I can have my own patched version of  
boost in my project this may save some serious head aches in the future.

Mike Jackson
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

troy d. straszheim
Michael Jackson wrote:

>
>
> On Dec 7, 2009, at 1:28 PM, troy d. straszheim wrote:
>
>> Michael Jackson wrote:
>>> So you are wanting to include the Boost sources in your project and
>>> you just want to build a specific subset of Boost to use with your
>>> project?
>>
>> Here's what I came up with:
>>
>> http://sodium.resophonic.com/boost-cmake/current-docs/exported_targets.html#with-boost-source-in-a-subdirectory-of-your-source 
>>
>>
>> the EXCLUDE_FROM_ALL option to add_subdirectory makes things
>> remarkably easy... nice feature.
>>
>> -t
>>
>>
>
>
> Nice,
>    Couple of comments though:
> "Setting BUILD_PROJECTS, etc is unnecessary, but if all the extra boost
> targets bother you you can set it at the commandline as usual:"
>
> Why can't I set BUILD_PROJECTS in my own CMakeLists.txt file? Wouldn't
> this get rid of the need to pass this on the command line? Also what
> about those running CMake-GUI? They don't have a command line to work from?

Yup, you can, it needs to be CACHE STRING FORCE.

>  By using the name of the target (boost_filesystem-mt-shared in your
> example), will CMake automatically "do the right thing" with respect to
> Debug and release libraries? Will building my project as a Debug pick up
> the Debug Boost libraries?

No, boost-cmake doesn't play nice with this kind of thing.  In the
beginning we were attempting to do things the way bjam does them,
compiling all variants with one command.  In retrospect I think this was
a really bad idea.  Now things clash, and it would be major surgery to
fix... major surgery that I may yet do, but realize that it would break
a growing amount of code that depends on the way boost-cmake now does
things.  :(

You could check CMAKE_BUILD_TYPE and tweak the name that you're linking
against:

if (CMAKE_BUILD_TYPE STREQUAL Debug)
   set(DBG_OR_RELEASE "-debug")
endif()

target_link_libraries(mything boost_filesystem-mt-shared${DBG_OR_RELEASE})

>  I see this as having some issues? I now have to detect when someone
> wants a "Debug" build then set the "BOOST_ENABLE_DEBUG=ON" before
> calling the add_subdirectory(boost-src).

If you just let boost.cmake configure all the targets, use
EXCLUDE_FROM_ALL and tweak the name you link to above, only what you
need gets built, and your CMAKE_BUILD_TYPE works as you expect it to.
The same name-tweaking above will work if you use exported targets
instead of boost-src-in-subdirectory.

>  I say all this without having tried any of this but it _is_ intriguing
> me currently due to all the problems trying to get Boost built OS X and
> Windows without memory leaks, compile errors and, ya know, basic stuff
> like that. If I can have my own patched version of boost in my project
> this may save some serious head aches in the future.

And on the boost-cmake list we should talk about what those patches
would be.  You're not the only one doing this, and we have the capacity
to do point releases of boost asynchronously from upstream.

-t





_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

Brian Davis-2
See below

On Mon, Dec 7, 2009 at 1:26 PM, troy d. straszheim <[hidden email]> wrote:
Michael Jackson wrote:


On Dec 7, 2009, at 1:28 PM, troy d. straszheim wrote:

Michael Jackson wrote:
So you are wanting to include the Boost sources in your project and you just want to build a specific subset of Boost to use with your project?

Here's what I came up with:

http://sodium.resophonic.com/boost-cmake/current-docs/exported_targets.html#with-boost-source-in-a-subdirectory-of-your-source

the EXCLUDE_FROM_ALL option to add_subdirectory makes things remarkably easy... nice feature.

-t

Yes this appears to be the closest thing to what I would like to do.

 


Nice,
  Couple of comments though:
"Setting BUILD_PROJECTS, etc is unnecessary, but if all the extra boost targets bother you you can set it at the commandline as usual:"

Why can't I set BUILD_PROJECTS in my own CMakeLists.txt file? Wouldn't this get rid of the need to pass this on the command line? Also what about those running CMake-GUI? They don't have a command line to work from?

Yup, you can, it needs to be CACHE STRING FORCE.


I am already taking advantage of CACHE STRING FORCE for multiple parameters to provide "checkout and build" automation with out having to specify these variables each time.  I only wish I could do this for the text box "Where to build the Binaries" i.e. the ability to fill this in with a default as I build out of source, but within the checkout directory structure ( rootofcheckout/source and rootofcheckout/build).  This way it would be a 3 step to development for developers 1) checkout tree and 2) point cmake to CMakeLists.txt and 3) press go.
 

 By using the name of the target (boost_filesystem-mt-shared in your example), will CMake automatically "do the right thing" with respect to Debug and release libraries? Will building my project as a Debug pick up the Debug Boost libraries?

No, boost-cmake doesn't play nice with this kind of thing.  In the beginning we were attempting to do things the way bjam does them, compiling all variants with one command.  In retrospect I think this was a really bad idea.  Now things clash, and it would be major surgery to fix... major surgery that I may yet do, but realize that it would break a growing amount of code that depends on the way boost-cmake now does things.  :(

You could check CMAKE_BUILD_TYPE and tweak the name that you're linking against:

if (CMAKE_BUILD_TYPE STREQUAL Debug)
 set(DBG_OR_RELEASE "-debug")
endif()

target_link_libraries(mything boost_filesystem-mt-shared${DBG_OR_RELEASE})


The do the right thing feature of bjam was quite nice.  This is what I was looking for in CMake as well.  It sounds from this discussion that it is a boost usage of CMake issue not a CMake issue correct or no?  Not sure this is a "solution".
 

 
 I see this as having some issues? I now have to detect when someone wants a "Debug" build then set the "BOOST_ENABLE_DEBUG=ON" before calling the add_subdirectory(boost-src).

If you just let boost.cmake configure all the targets, use EXCLUDE_FROM_ALL and tweak the name you link to above, only what you need gets built, and your CMAKE_BUILD_TYPE works as you expect it to. The same name-tweaking above will work if you use exported targets instead of boost-src-in-subdirectory.


ouch... ok will keep this in mind.
 
 I say all this without having tried any of this but it _is_ intriguing me currently due to all the problems trying to get Boost built OS X and Windows without memory leaks, compile errors and, ya know, basic stuff like that. If I can have my own patched version of boost in my project this may save some serious head aches in the future.
 

And on the boost-cmake list we should talk about what those patches would be.  You're not the only one doing this, and we have the capacity to do point releases of boost asynchronously from upstream.

-t






_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake



--
Brian J. Davis


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Build only what you need in third party libs

Alexander Neundorf-3
In reply to this post by Brian Davis-2
On Saturday 05 December 2009, Brian Davis wrote:

> I have used boost jam before and there was a mechanism to build only what
> you need by specifying dependencies within the boost libraries.  bjam would
> then run off calculate deps and build only what I needed. In CMake is there
> such a feature to specify a third party library and specify only the
> targets you need to build as dependencies of my build target?
>
> i.e. can I use
>
> add_subdirectory( lib/3rdParty/Win32/boost-cmake-1_41_0 ./boost )
>
> then create a target that only uses say boost_filesystem and only have that
> built? Or do I need to build the world to get a handful of needed dll's
> that are actually used in my project?

Maybe I'm missing the point, but let's say you have library A, B, C and D
(where B depends on A), and an executable foo, which links against B and D.
If you then just build the target "foo", the targets A, B and D will be built
automatically (and not C, since foo does not depend on C).

Is this what you mean ?

Alex
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake