Embedding up-to-date version info in built library

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Embedding up-to-date version info in built library

Kolja Waschk
Hi,

I know that similar questions have been posted before, but as far as I
have been able to follow, no real solution was ever given.

The goal is to embed revision information in a C++ library "mylib".

I already managed to use "create_custom_target" so that an external
program is executed to create a "version.cc" file each time that "make
mylib" (using GNU make) is called. The corresponding object is
re-compiled and the library is put together from this plus 100 other
object files.

However, collecting all these object files takes some time, especially
on a Windows host. Within the same source tree, we build applications
using the library. Due to the forced version.cc update, this time is
always taken, regardless whether only application code or library code
changed. Now I tried hard to make my custom target (or custom command)
dependend only on changes to the library code, but failed so far.

Assume I have a list of all *.cc source files in variable SOURCES:

Try #1: add_custom_command(OUTPUT version.cc DEPENDS ${SOURCES} ...)

This isn't sufficient because it would ignore changes to the header
files that ${SOURCES} depend on.

Try #2: as above plus IMPLICIT_DEPENDS CXX ${SOURCES} ...

As someone else here noted just a few days ago, the IMPLICIT_DEPENDS
seems to work only if the ${SOURCES} come with full path info. Actually,
it started to work after I added the path info in a foreach() loop to
the list of sources. But another problem came up: Only the first single
list item seems to be scanned for implicit dependencies. And I failed to
come up with a list or command syntax that would allow me to have the
other 99+ sources scanned as well.

Try #3: gather the list of dependencies with output_required_files().

It failed because output_required_files() seems to output empty files in
any case, regardless of how input files are specified.

Try #4: Build another library without the version information, make
a custom_command to update version.cc with that library as a target and
for POST_BUILD, and then create the actual library with version
information dependent on the other library. That works, but now
of course all the object files are compiled twice. But no library is
recompiled if just app code changed. The app developer is happy while
the lib developer gets his sleep ;)

Now if you have any idea to workaround any of the problems described
above, I't really appreciate your help! The solution should not be
limited to Unix systems. I'm currently using CMake 2.6.4.

Kolja

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Kolja Waschk
> the list of sources. But another problem came up: Only the first single
> list item seems to be scanned for implicit dependencies. And I failed to
> come up with a list or command syntax that would allow me to have the
> other 99+ sources scanned as well.

I think I came up with a usable workaround for now. A semaphore file is
created for every source file which IMPLICIT_DEPENDS on that source
file, and my actual version.cc now depends on all the semaphore files.

Basically this:


foreach (srcfile ${SOURCEFILES})
   string(REPLACE "/" "_" srcok ${srcfile})
   set(semafile dep_${srcok})
   add_custom_command(
     OUTPUT ${semafile}
     COMMAND touch ${semafile}
     IMPLICIT_DEPENDS CXX ${PROJECT_SOURCE_DIR}/src/${srcfile}
     COMMENT ""
   )
   set(VERSION_DEPFILES ${VERSION_SEMAFILES} ${semafile})
endforeach(srcfile)

add_custom_command(
   OUTPUT version.cc
   COMMAND create_version_cc.sh
   DEPENDS ${VERSION_SEMAFILES}
)

I'm open for improvements and better ideas...

Kolja



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Michael Wild
In reply to this post by Kolja Waschk

On 11. Dec, 2009, at 15:53 , Kolja Waschk wrote:

> Hi,
>
> I know that similar questions have been posted before, but as far as I
> have been able to follow, no real solution was ever given.
>
> The goal is to embed revision information in a C++ library "mylib".
>
> I already managed to use "create_custom_target" so that an external
> program is executed to create a "version.cc" file each time that "make
> mylib" (using GNU make) is called. The corresponding object is
> re-compiled and the library is put together from this plus 100 other
> object files.
>
> However, collecting all these object files takes some time, especially
> on a Windows host. Within the same source tree, we build applications
> using the library. Due to the forced version.cc update, this time is
> always taken, regardless whether only application code or library code
> changed. Now I tried hard to make my custom target (or custom command)
> dependend only on changes to the library code, but failed so far.
>
> Assume I have a list of all *.cc source files in variable SOURCES:
>
> Try #1: add_custom_command(OUTPUT version.cc DEPENDS ${SOURCES} ...)
>
> This isn't sufficient because it would ignore changes to the header
> files that ${SOURCES} depend on.
>
> Try #2: as above plus IMPLICIT_DEPENDS CXX ${SOURCES} ...
>
> As someone else here noted just a few days ago, the IMPLICIT_DEPENDS
> seems to work only if the ${SOURCES} come with full path info. Actually,
> it started to work after I added the path info in a foreach() loop to
> the list of sources. But another problem came up: Only the first single
> list item seems to be scanned for implicit dependencies. And I failed to
> come up with a list or command syntax that would allow me to have the
> other 99+ sources scanned as well.
>
> Try #3: gather the list of dependencies with output_required_files().
>
> It failed because output_required_files() seems to output empty files in
> any case, regardless of how input files are specified.
>
> Try #4: Build another library without the version information, make
> a custom_command to update version.cc with that library as a target and
> for POST_BUILD, and then create the actual library with version
> information dependent on the other library. That works, but now
> of course all the object files are compiled twice. But no library is
> recompiled if just app code changed. The app developer is happy while
> the lib developer gets his sleep ;)
>
> Now if you have any idea to workaround any of the problems described
> above, I't really appreciate your help! The solution should not be
> limited to Unix systems. I'm currently using CMake 2.6.4.
>
> Kolja

One question: Why do you need a add_custom_command with all those dependencies? Why does version.cc depend on all the source and header files? What do you write in your version.cc file? Is it something like this:


/*       version.cc                    */
const char* const VERSION = "10.5.1";
/*       end of version.cc             */


If so, you don't need such an elaborate scheme, a file like this will do:

/*       version.cc.in                 */
const char* const VERSION = "@VERSION@";
/*       end of version.cc.in          */


and then in your CMake code you do:

# CMakeLists.txt
# ... a lot of code ...
set(VERSION "10.5.1")
configure_file(version.cc.in ${CMAKE_BINARY_DIR}/version.cc @ONLY)

add_library(something src1 src2 src3 ${CMAKE_BINARY_DIR}/version.cc)
# ... some more code ...


Michael
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Kolja Waschk
Hi, thanks,

> One question: Why do you need a add_custom_command with all those
> dependencies? Why does version.cc depend on all the source and header
> files? What do you write in your version.cc file? Is it something like

Beside the SVN revision number of the working base, our embedded
revision information also tells whether the library was built from code
with local modifications, i.e. whether it equals the source code stored
in the SVN repository or was changed afterwards.  Changing a single byte
in a header file already affects that state, so all headers need to be
watched.

Kolja
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Michael Wild

On 11. Dec, 2009, at 16:58 , Kolja Waschk wrote:

> Hi, thanks,
>
>> One question: Why do you need a add_custom_command with all those
>> dependencies? Why does version.cc depend on all the source and header
>> files? What do you write in your version.cc file? Is it something like
>
> Beside the SVN revision number of the working base, our embedded
> revision information also tells whether the library was built from code
> with local modifications, i.e. whether it equals the source code stored
> in the SVN repository or was changed afterwards.  Changing a single byte
> in a header file already affects that state, so all headers need to be
> watched.
>
> Kolja

I made some experiments, and I can confirm your observation: only the first file in the IMPLICIT_DEPENDS list gets added. Further, the documentation is inaccurate and doesn't mention that the language has to be prepended to every single file (i.e. <lang> <file> pairs).

Currently I don't quite understand how the whole thing works...

Michael
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Brad King
In reply to this post by Kolja Waschk
Kolja Waschk wrote:
> Beside the SVN revision number of the working base, our embedded
> revision information also tells whether the library was built from code
> with local modifications, i.e. whether it equals the source code stored
> in the SVN repository or was changed afterwards.  Changing a single byte
> in a header file already affects that state, so all headers need to be
> watched.

So you need some combination of "svn info" and "svn status".

> Try #1: add_custom_command(OUTPUT version.cc DEPENDS ${SOURCES} ...)

You need the custom command to run every time the build starts:

  add_custom_target(update_version
    COMMAND ${CMAKE_COMMAND} -P update_version.cmake
    )

The update_version.cmake script should compute the version and then
use configure_file() to store the result in "version.h" with a
copy-if-different check.  The key is then to make the library depend
on this rule at a *target* level, but only on version.h at a *file*
level.

  add_library(mylib version.c) # version.c does #include "version.h"
  add_dependencies(mylib update_version)

The update_version target will always run the command before the build
system considers mylib.  Then the version.c source will recompile only
if the version update rule actually changed version.h content.

> Try #2: as above plus IMPLICIT_DEPENDS CXX ${SOURCES} ...

FYI, any solution based on IMPLICIT_DEPENDS will not work with VS IDE
and other non-Makefile generators.  It is documented to work only with
Makefile generators.

-Brad
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Kolja Waschk
Hi Brad,

> You need the custom command to run every time the build starts:
...
> copy-if-different check.  The key is then to make the library depend
> on this rule at a *target* level, but only on version.h at a *file* level.

Because such a custom target is "always considered out of date", the library
that depends on it is also considered out of date and would be rebuilt, even
though no source code changed. Assembling our 100+ object files into a large
library is fast on Linux but dead slow on Windows/Cygwin. At least it is much
slower than reading and comparing the timestamps of the 100+ files (no,
disabling the virus scanner didn't help much;)

From the fact that it is possible and simple to define library targets that
depend on object files that depend on source code that depend on header files,
I hoped that it would be just easy to have arbitrary other targets depend
on similarly complex input (in my case: the same minus one file). I put some
hope in the "output_required_files()" command but it always outputs nil.

Kolja


--
mr. kolja waschk - haubach-39 - 22765 hh - germany
fon +49 40 889130-34 - fax -35 - http://www.ixo.de

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Brad King
Waschk,Kolja wrote:

> Hi Brad,
>
>> You need the custom command to run every time the build starts:
> ...
>> copy-if-different check.  The key is then to make the library depend
>> on this rule at a *target* level, but only on version.h at a *file*
>> level.
>
> Because such a custom target is "always considered out of date", the
> library that depends on it is also considered out of date and would
> be rebuilt,

That's why the key is to avoid the *file* level dependency.  The *target*
level dependency only determines the order in which the build system
evaluates targets.  Within each target the file-level dependencies are
separately evaluated.

This code:

  add_custom_target(update_version COMMAND echo updating)
  add_library(mylib version.c)
  add_dependencies(mylib update_version)

causes "updating" to print every time you type "make", but it will
build mylib only when sources change.  It works in the VS IDE too.

-Brad
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedding up-to-date version info in built library

Kolja Waschk
Hi,

>  add_custom_target(update_version COMMAND echo updating)
>  add_library(mylib version.c)
>  add_dependencies(mylib update_version)

We had exactly this configuration for months and the lib was rebuilt
always... but now that you affirm that it /should/ work better, I
had a deeper look again. The actual cause for the rebuild is the
compile time embedded in version.c. The updater script doesn't check
dependencies. It just computes a new version.c and that new version.c
will always be different from the previous one because the time
changes. Unless we decide to omit that time info, we need to make
updates of version.c dependent on whether the library should be rebuild
for other reasons. So, we really need to reconsider whether the time info in
that file makes much sense...

Thanks for your help!
Kolja

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake