Generated source files and dependencies(+) (Wojciech Migda)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Generated source files and dependencies(+) (Wojciech Migda)

Wojciech Migda-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

some time ago there was a discussion, which has highlighted problem I
am facing now, but it did not provide final conclusions
(http://www.cmake.org/pipermail/cmake/2008-September/024093.html).

Take this example:

- --------------------------
INCLUDE_DIRECTORIES( ${CMAKE_CURRENT_SOURCE_DIR} )

SET_SOURCE_FILE_PROPERTIES( ${CMAKE_CURRENT_SOURCE_DIR}/a.h PROPERTIES
GENERATED TRUE )

ADD_CUSTOM_COMMAND(
  OUTPUT "${CMAKE_CURRENT_SOURCE_DIR}/a.h"
  COMMAND echo \"\#define STAMP \\\"\" `date` \"\\\"\"  > a.h
  DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/a.txt"
)

ADD_CUSTOM_TARGET( a_h_gen DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/a.h )

ADD_LIBRARY( foo STATIC a.c )
- ---------------------------------

Here we have file a.c which includes file a.h.
a.h is generated, with dependency on a.txt.

What I'd expect from the above (and what ZNV hinted in the 2008
thread) building foo library should check dependencies of a.c and if
a.h doesn't exist generate it or it a.txt has changed regenerate a.h

However, files generated with cmake 2.6.4 show that:

1. in Makefile2 a.c has no dependencies on a.h or a_h_gen
2. depend.make for foo library has dependency on a.h only if a.h
existed at the time of cmake execution or during dependency scanning
for a.c

The result is that building foo doesn't generate a.h nor a.h is
refreshed if a.txt changed even though depend.make for foo lists a.h
as its dependency. If I add the following line at the end:

ADD_DEPENDENCIES( foo a_h_gen )

a dependency for foo on a_h_gen will appear in Makefile2 and
everything will work as expected.

This however, in a real life example I am working on there are
hundreds of source files and tens of libraries which indirectly may
depend on a generated header and thus manual specification of
dependency between built library and generated file is simply impractical.

Thus, my question is whether I am missing something in my example
CMakeLists.txt or cmake at this stage is simply unable to handle such
situation.

In the 2008 discussion ZNV illustrated exactly the same situation by
the single vs. two-level dependency illustration, and Esben's response
was that properly constructed CMakeLists.txt file shall enforce single
level of dependencies. Nonetheless, the example above shows that it is
not the case.

Any hints allowing me to reach the desired cmake behaviour will be
appreciated. THANKS.

- -Wojciech
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKprET0iFl+nAyImcRAritAJ94VawRLcOh55U6uBu+QECWbVKB7wCffST5
TYAJVZC8mGhNp3yO9fVue2Y=
=LYc3
-----END PGP SIGNATURE-----


----------------------------------------------------------------------
Wykonaj diagnostyke skory i odbierz swoj Prezent!
http://link.interia.pl/f22f1

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Clinton Stimpson

Why not include it in the foo target, instead of making a new a_h_gen target
and doing extra dependencies manually?

-----------------
INCLUDE_DIRECTORIES( ${CMAKE_CURRENT_SOURCE_DIR} )
SET_SOURCE_FILE_PROPERTIES( ${CMAKE_CURRENT_SOURCE_DIR}/a.h PROPERTIES    
   GENERATED TRUE )

ADD_CUSTOM_COMMAND(
  OUTPUT "${CMAKE_CURRENT_SOURCE_DIR}/a.h"
  COMMAND echo \"\#define STAMP \\\"\" `date` \"\\\"\"  > a.h
  DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/a.txt"
)

ADD_LIBRARY( foo STATIC a.c ${CMAKE_CURRENT_SOURCE_DIR}/a.h)
----------------

Clint

On Tuesday 08 September 2009 01:31:32 pm Wojciech Migda wrote:

> Hi,
>
> some time ago there was a discussion, which has highlighted problem I
> am facing now, but it did not provide final conclusions
> (http://www.cmake.org/pipermail/cmake/2008-September/024093.html).
>
> Take this example:
>
> --------------------------
> INCLUDE_DIRECTORIES( ${CMAKE_CURRENT_SOURCE_DIR} )
>
> SET_SOURCE_FILE_PROPERTIES( ${CMAKE_CURRENT_SOURCE_DIR}/a.h PROPERTIES
> GENERATED TRUE )
>
> ADD_CUSTOM_COMMAND(
>   OUTPUT "${CMAKE_CURRENT_SOURCE_DIR}/a.h"
>   COMMAND echo \"\#define STAMP \\\"\" `date` \"\\\"\"  > a.h
>   DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/a.txt"
> )
>
> ADD_CUSTOM_TARGET( a_h_gen DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/a.h )
>
> ADD_LIBRARY( foo STATIC a.c )
> ---------------------------------
>
> Here we have file a.c which includes file a.h.
> a.h is generated, with dependency on a.txt.
>
> What I'd expect from the above (and what ZNV hinted in the 2008
> thread) building foo library should check dependencies of a.c and if
> a.h doesn't exist generate it or it a.txt has changed regenerate a.h
>
> However, files generated with cmake 2.6.4 show that:
>
> 1. in Makefile2 a.c has no dependencies on a.h or a_h_gen
> 2. depend.make for foo library has dependency on a.h only if a.h
> existed at the time of cmake execution or during dependency scanning
> for a.c
>
> The result is that building foo doesn't generate a.h nor a.h is
> refreshed if a.txt changed even though depend.make for foo lists a.h
> as its dependency. If I add the following line at the end:
>
> ADD_DEPENDENCIES( foo a_h_gen )
>
> a dependency for foo on a_h_gen will appear in Makefile2 and
> everything will work as expected.
>
> This however, in a real life example I am working on there are
> hundreds of source files and tens of libraries which indirectly may
> depend on a generated header and thus manual specification of
> dependency between built library and generated file is simply impractical.
>
> Thus, my question is whether I am missing something in my example
> CMakeLists.txt or cmake at this stage is simply unable to handle such
> situation.
>
> In the 2008 discussion ZNV illustrated exactly the same situation by
> the single vs. two-level dependency illustration, and Esben's response
> was that properly constructed CMakeLists.txt file shall enforce single
> level of dependencies. Nonetheless, the example above shows that it is
> not the case.
>
> Any hints allowing me to reach the desired cmake behaviour will be
> appreciated. THANKS.
>
> -Wojciech
>
>
> ----------------------------------------------------------------------
> Wykonaj diagnostyke skory i odbierz swoj Prezent!
> http://link.interia.pl/f22f1
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Wojciech Migda-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Why not include it in the foo target, instead of making a new
a_h_gen target and doing extra dependencies manually?

Firstly, we have hundred of source files which may indirectly depend
generated source files, so we want such information to be covered by
cmake dependency scanner itself - the problem is that there is no link
between the library target and the header target.

By no means we want to specify such dependencies manually - that would
be a nightmare.

If we skip the a_h_gen target the header generation target will not
appear in Makefile2, which I think is one of the required links for
everything to work. The last remaining link is missing (which we may
mimic by hand with the add_dependencies command) by I don't know how
to fix it so it becomes automatic within the build system and
dependency scanner.

- -Wojciech
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKprs10iFl+nAyImcRAo5XAJ4scpKe/E9g5uTZcFPyLYroJmD0oACghdom
T6TR/kPv+kLYga/wCXObbwA=
=q652
-----END PGP SIGNATURE-----



----------------------------------------------------------------------
Marcin Gortat – gwiazda NBA w naszej reprezentacji!
Czytaj wiecej >> http://link.interia.pl/f232a

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Clinton Stimpson
On Tuesday 08 September 2009 02:14:45 pm Wojciech Migda wrote:
> > Why not include it in the foo target, instead of making a new
>
> a_h_gen target and doing extra dependencies manually?
>
> Firstly, we have hundred of source files which may indirectly depend
> generated source files, so we want such information to be covered by
> cmake dependency scanner itself - the problem is that there is no link
> between the library target and the header target.

That's why I suggested
ADD_LIBRARY(foo STATIC a.c ${CMAKE_CURRENT_BINARY_DIR}/a.h)

It creates the generated headers for foo, then does the dependency scanning
for foo, then compiles files.

Here's what I got:
$ make
[ 50%] Generating a.h
Scanning dependencies of target foo
[100%] Building C object CMakeFiles/foo.dir/a.o
Linking C static library libfoo.a

If that doesn't work for your case, can you be more specific on why it doesn't?

Clint

>
> By no means we want to specify such dependencies manually - that would
> be a nightmare.
>
> If we skip the a_h_gen target the header generation target will not
> appear in Makefile2, which I think is one of the required links for
> everything to work. The last remaining link is missing (which we may
> mimic by hand with the add_dependencies command) by I don't know how
> to fix it so it becomes automatic within the build system and
> dependency scanner.
>
> -Wojciech
>
>
>
> ----------------------------------------------------------------------
> Marcin Gortat – gwiazda NBA w naszej reprezentacji!
> Czytaj wiecej >> http://link.interia.pl/f232a
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Wojciech Migda-3
In reply to this post by Wojciech Migda-3
Użytkownik "Clinton Stimpson"  napisał(a):
> From: "Clinton Stimpson"
> Subject: Re: [CMake] Generated source files and dependencies(+)
(Wojciech Migda)

> To: [hidden email]
>
> On Tuesday 08 September 2009 02:14:45 pm Wojciech Migda wrote:
> > > Why not include it in the foo target, instead of making a new
> >
> > a_h_gen target and doing extra dependencies manually?
> >
> > Firstly, we have hundred of source files which may indirectly depend
> > generated source files, so we want such information to be covered by
> > cmake dependency scanner itself - the problem is that there is no
link

> > between the library target and the header target.
>
> That's why I suggested
> ADD_LIBRARY(foo STATIC a.c ${CMAKE_CURRENT_BINARY_DIR}/a.h)
>
> It creates the generated headers for foo, then does the dependency
> scanning
> for foo, then compiles files.
>
> Here's what I got:
> $ make
> [ 50%] Generating a.h
> Scanning dependencies of target foo
> [100%] Building C object CMakeFiles/foo.dir/a.o
> Linking C static library libfoo.a
>
> If that doesn't work for your case, can you be more specific on why it
> doesn't?
>

hi, your proposal suggests implicit addition of dependency on a.h for the
library. We have tens of libraries, some of which might need dependency on
a.h, some might not. If we add this dependency on those which do not need
it then they will be rebuilt unnecesarilly if the a.txt changes.

What I'd like to see is cmake take care of the dependencies in a way that
when I type 'make foo' and a.txt is changed, then a.h would be
regenerated/created and foo rebuilt with new a.h. I guess cmake has got all
necessary information to convey such operation, since it does scanning of
dependencies on a.c to know its dependency on a.h, it knows that there is a
rule for a.h generation, so why not combine the two automaticaly and make
it work together ?

-Wojciech




----------------------------------------------------------------------
Lecza czy truja? Poczytaj o niezwyklych wlasciwosciach grzybow.
Sprawdz >>> http://link.interia.pl/f232b

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Clinton Stimpson
In reply to this post by Wojciech Migda-3
On 09/09/2009 01:15 AM, Wojciech Migda wrote:

> Użytkownik "Clinton Stimpson"  napisał(a):
>
>> From: "Clinton Stimpson"
>> Subject: Re: [CMake] Generated source files and dependencies(+) (Wojciech Migda)
>> To: [hidden email]
>>
>> On Tuesday 08 September 2009 02:14:45 pm Wojciech Migda wrote:
>>
>>>> Why not include it in the foo target, instead of making a new
>>>>
>>> a_h_gen target and doing extra dependencies manually?
>>>
>>> Firstly, we have hundred of source files which may indirectly depend
>>> generated source files, so we want such information to be covered by
>>> cmake dependency scanner itself - the problem is that there is no link
>>> between the library target and the header target.
>>>
>> That's why I suggested
>> ADD_LIBRARY(foo STATIC a.c ${CMAKE_CURRENT_BINARY_DIR}/a.h)
>>
>> It creates the generated headers for foo, then does the dependency
>> scanning
>> for foo, then compiles files.
>>
>> Here's what I got:
>> $ make
>> [ 50%] Generating a.h
>> Scanning dependencies of target foo
>> [100%] Building C object CMakeFiles/foo.dir/a.o
>> Linking C static library libfoo.a
>>
>> If that doesn't work for your case, can you be more specific on why it
>> doesn't?
>>
>>
> hi, your proposal suggests implicit addition of dependency on a.h for the library. We have tens of libraries, some of which might need dependency on a.h, some might not.
Then use target_link_libraries() / add_dependency() between libraries,
so foo gets built before.
>   If we add this dependency on those which do not need it then they will be rebuilt unnecesarilly if the a.txt changes.
>
Yeah, you should only add a.h to one library/target, and let the others
depend on it.
> What I'd like to see is cmake take care of the dependencies in a way that when I type 'make foo' and a.txt is changed, then a.h would be regenerated/created and foo rebuilt with new a.h. I guess cmake has got all necessary information to convey such operation, since it does scanning of dependencies on a.c to know its dependency on a.h, it knows that there is a rule for a.h generation, so why not combine the two automaticaly and make it work together ?
>

And be sure your add_custom_command() has the DEPENDS option to depend
on a.txt, so a.h will regenerate when a.txt changes.

The dependency scanning will pick up a.h for other targets/libraries,
even if they are in different directories.

Clint

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Alexander Neundorf-3
In reply to this post by Wojciech Migda-3
On Tuesday 08 September 2009, Wojciech Migda wrote:

> > Why not include it in the foo target, instead of making a new
>
> a_h_gen target and doing extra dependencies manually?
>
> Firstly, we have hundred of source files which may indirectly depend
> generated source files, so we want such information to be covered by
> cmake dependency scanner itself - the problem is that there is no link
> between the library target and the header target.
>
> By no means we want to specify such dependencies manually - that would
> be a nightmare.

If you mean by specifying manually adding the generated header files manually
to the targets, this can be made much easier with the support of some macro:

macro(generate_stuff srcs )
   ...
   add_custom_command(OUTPUT foo.h ...)
   ...
   set( ${srcs} ${${srcs}} foo.h)
endmacro()


set(mySrcs main.cpp bar.cpp)
generate_stuff(mySrcs template1.xml template2.xml)

add_executable(hello ${mySrcs} )

We are using that a lot e.g. in KDE4.

Alex
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generated source files and dependencies(+) (Wojciech Migda)

Wojciech Migda-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Alexander Neundorf pisze:

> On Tuesday 08 September 2009, Wojciech Migda wrote:
>>> Why not include it in the foo target, instead of making a new
>> a_h_gen target and doing extra dependencies manually?
>>
>> Firstly, we have hundred of source files which may indirectly depend
>> generated source files, so we want such information to be covered by
>> cmake dependency scanner itself - the problem is that there is no link
>> between the library target and the header target.
>>
>> By no means we want to specify such dependencies manually - that would
>> be a nightmare.
>
> If you mean by specifying manually adding the generated header files
manually
> to the targets, this can be made much easier with the support of some
macro:

>
> macro(generate_stuff srcs )
>    ...
>    add_custom_command(OUTPUT foo.h ...)
>    ...
>    set( ${srcs} ${${srcs}} foo.h)
> endmacro()
>
>
> set(mySrcs main.cpp bar.cpp)
> generate_stuff(mySrcs template1.xml template2.xml)
>
> add_executable(hello ${mySrcs} )
>
> We are using that a lot e.g. in KDE4.

But this implies that we know beforehand that certain library has
dependency on the generated header or not and that we will construct
the CMakeLists.txt with that in mind. This makes the whole thing not
much different than doing dependency scan of headers outside cmake and
embedding the results by hand. If during the course of development the
dependency of a certain target on a.h will change the contents of the
CMakeLists.txt would have to be changed as well - I don't take this as
an elegant solution, to mention maintenance efforts only.

Isn't there any way to enable cmake to scan this dependency
automatically ?

I even considered using OUTPUT_REQUIRED_FILES to obtain information
whether certain files would have dependency on the generated header to
conditionally enable dependency on it. But this would be yet another
workaround to achieve something I'd expect from this kind of tool as
cmake is to be done transparently. Frankly, I'm a bit surprised that
cmake hasn't got such feature yet. Or I'm just wrong ?

Thanks in advance,

- -Wojciech
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKp+n60iFl+nAyImcRAtGbAJsFNnvRbljipG9VGL88jyrhAoLFpwCePV18
436+WsTsX8daghTVTGtQhjc=
=WLqz
-----END PGP SIGNATURE-----


----------------------------------------------------------------------
Praca za granic±? Zobacz oferty!
http://link.interia.pl/f2331

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

OUTPUT_REQUIRED_FILES - how does it work ?

Wojciech Migda-3
In reply to this post by Clinton Stimpson
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi again,

while exploring possibilities to solve dependency problems I took a
peek at the OUTPUT_REQUIRED_FILES command.
I constructed simple CMakeLists.txt

OUTPUT_REQUIRED_FILES( bar.c baz.txt )

ADD_LIBRARY( foo bar.c )


and the contents of bar.c is:

#include <stdio.h>

int main()
{
    puts("hello world!");
    return 0;
}

after running cmake baz.txt is created, but its empty. Building the
library with make gives no effect neither. I tried to run
OUTPUT_REQUIRED_FILES on bar.o (depend.make contains dependencies on
bar.o) but it didn't help.

Can anyone point me in the right direction ?

Thanks in advance,

- -Wojciech
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKqUE+0iFl+nAyImcRAt4kAJ4lqlSb1Nm+TQJrkwCuT8n/9szy0ACfUslV
H7NgAJIUnSNPr5YDgsj3Vy0=
=thTx
-----END PGP SIGNATURE-----


----------------------------------------------------------------------
Wygraj telefon HTC Touch Diamond 2.
Sprawd� >> http://link.interia.pl/f233f



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake