CMake copy files when they change to build dir

[Eric Jonas]

Hello,

I use CMake with boost::python to generate python modules, and it does a
great job of creating the required .so files. There's an interesting
question, however, of how to test the resulting object. When doing an
out-of-source build, I end up with

build/myproj/foo.so

and in my source tree I have

src/myproj/test_foo.py

I'd like to, as part of the build process, copy  test_foo.py
to build/myproj/test_foo.py . That is, when I type "make", if
src/myproj/test_foo.py differs from build/myproj/test_foo.py, copy it
over.

I've tried various incantations of add_custom_command but haven't been
able to figure out the right thing to do. It seems like somehow
I want the underlying generated Makefile to contain the relevant "cp"
command, but thus far I have been unsuccessful.

Thanks for any help you can provide,
                        ...Eric




_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Tyler Roscoe]

On Fri, Aug 07, 2009 at 10:08:16AM -0400, Eric Jonas wrote:
> I'd like to, as part of the build process, copy  test_foo.py
> to build/myproj/test_foo.py . That is, when I type "make", if
> src/myproj/test_foo.py differs from build/myproj/test_foo.py, copy it
> over.
>
> I've tried various incantations of add_custom_command but haven't been
> able to figure out the right thing to do. It seems like somehow
> I want the underlying generated Makefile to contain the relevant "cp"
> command, but thus far I have been unsuccessful.

There is a FAQ entry about this that ought to work. Why don't you post
what you have so we can see where you're going wrong?

Also you can use cmake -E copy_if_different as the command part. It is
platform-independent, unlike cp.

tyler
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Eric Jonas]

>
> There is a FAQ entry about this that ought to work. Why don't you post
> what you have so we can see where you're going wrong?
>
> Also you can use cmake -E copy_if_different as the command part. It is
> platform-independent, unlike cp.

Tyler, thanks for the quick response -- after quite a bit of frustrated
head-hitting-desk action, I came up with:


set(test_PYFILES
  test_basics.py
)
 
 
foreach(pyfile ${test_PYFILES})
  ADD_CUSTOM_COMMAND(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${pyfile}"
    COMMAND cmake -E copy "${CMAKE_CURRENT_SOURCE_DIR}/${pyfile}"
    "${CMAKE_CURRENT_BINARY_DIR}/${pyfile}"
    DEPENDS   "${CMAKE_CURRENT_SOURCE_DIR}/${pyfile}"
    )
   list(APPEND pyfile_dest "${CMAKE_CURRENT_BINARY_DIR}/${pyfile}")
 
endforeach(pyfile)
ADD_CUSTOM_TARGET(silly ALL DEPENDS ${pyfile_dest})

I was reading the FAQs too quickly, and missing the line

"""This tells CMake how to build the file but does not actually add a
rule to the build system. Another target must require it. One may create
a custom target explicitly for this rule:

ADD_CUSTOM_TARGET(driver ALL DEPENDS someoutput.txt)
"""

Regardless, things are working now, yay! Thanks again,
                        ...Eric



 


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[avner cohen]

Hi all,
 
I'm trying to set a rule/command/somthing, that will be executed with every build (even if cmake files are not touched) to check for DiskSpace available before starting the build.
In case of error, I'd like the build to exit ofcourse.
 
Any suggestions?
 
Regards,
 
        Avner Cohen.


     
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Eric Noulard]

2009/8/11 avner cohen <[hidden email]>:
>
> Hi all,
>
> I'm trying to set a rule/command/somthing, that will be executed with every build (even if cmake files are not touched) to check for DiskSpace available before starting the build.

I think you currently have no mean to ensure that some
command/target/something will be run first, however you can

1) create a custom target

    add_custom_target(CheckDisk  <your_check_disk_command>
                           COMMENT "Check available disk space before build")

2) then add this target as a dependency for ALL other target you add
ADD_LIBRARY, ADD_EXECUTABLE etc...

     add_executable(blah blah.c)
     add_dependencies(blah CheckDisk)

> In case of error, I'd like the build to exit ofcourse.

If the specified <your_check_disk_command> fails the build should stop.
Example project working with a pseudo checkdisk.sh, bash script attached.

Note that add_custom_target may use the ALL option which would force the target
to be run for EVERY build, but I don't know WHEN the target will be launched.

May be CMake developer can tell us when a ALLed add_custom_target will
be run? May be it will be run first if it is the first specified target in the
top level CMakeLists.txt??


--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[avner cohen]

Thank you very much Eric, this defiently helps.
My problem now is that I want my "<your_check_disk_command>" to be a MACRO/FUNCTION if already written in my cmake scripts (rather than running a shell script) is that possible ?

Thanks in advance,

-Avner.



----- Original Message ----
From: Eric Noulard <[hidden email]>
To: avner cohen <[hidden email]>
Cc: [hidden email]
Sent: Tuesday, August 11, 2009 10:16:29 AM
Subject: Re: [CMake] Checking Disk space with Cmake

2009/8/11 avner cohen <[hidden email]>:
>
> Hi all,
>
> I'm trying to set a rule/command/somthing, that will be executed with every build (even if cmake files are not touched) to check for DiskSpace available before starting the build.

I think you currently have no mean to ensure that some
command/target/something will be run first, however you can

1) create a custom target

    add_custom_target(CheckDisk  <your_check_disk_command>
                          COMMENT "Check available disk space before build")

2) then add this target as a dependency for ALL other target you add
ADD_LIBRARY, ADD_EXECUTABLE etc...

    add_executable(blah blah.c)
    add_dependencies(blah CheckDisk)

> In case of error, I'd like the build to exit ofcourse.

If the specified <your_check_disk_command> fails the build should stop.
Example project working with a pseudo checkdisk.sh, bash script attached.

Note that add_custom_target may use the ALL option which would force the target
to be run for EVERY build, but I don't know WHEN the target will be launched.

May be CMake developer can tell us when a ALLed add_custom_target will
be run? May be it will be run first if it is the first specified target in the
top level CMakeLists.txt??


--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org



     
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Eric Noulard]

2009/8/11 avner cohen <[hidden email]>:
>
> Thank you very much Eric, this defiently helps.
> My problem now is that I want my "<your_check_disk_command>" to be a MACRO/FUNCTION if already written in my cmake scripts (rather than running a shell script) is that possible ?

Almost yes, in fact you cannot "call a macro" in add_custom_target,
however you may put whatever you want in a checkdisk.cmake CMake script file
and then call this script  using CMake -P, like in:

add_custom_target(CheckDisk ${CMAKE_COMMAND} -P
${CMAKE_CURRENT_SOURCE_DIR}/checkdisk.cmake
                  COMMENT "Check available disk space before build"
                  SOURCES checkdisk.cmake)

If you need to pass value to your checkdisk.cmake, then you may use -D
arguments to CMake
and/or configure_file(checkdisk.cmake.in checkdisk.cmake)  in order to
make the called
script contains configured values from your project.


--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[avner cohen]

Hi Eric,
Thanks again for your help.
1. (Let me clarify here, that the Diskspace issue was a simple example of the wider solution I am looking for).
2. (Apologies for the long post).
 
As I know feel I've been fighting with this for long enough, I'll try again from scratch.
 
All in all, my feeling is that I'm inventing the wheel here as what I need to do seems rather basic (or maybe not?)
So I'll try to detail this from the beginning, maybe a new suggestion will come:
 
1. I have a series of NON cpp files (IDL files) that requires automatic generation whenever an IDL is being changed.
I was able to create the script to generate the IDLs:
EXECUTE_PROCESS (COMMAND ${IDL_COMPILER} -I ${PROJECT_SOURCE_DIR}IDLs ${PROJECT_SOURCE_DIR}/IDLs/CORBAStructDefs.idl WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/IDLs RESULT_VARIABLE  retcode)
 
2. I need to execute the above command ONLY when the IDL is changed (otherwise full build is incorrectly triggered each time), so I took the approach of creating a checksum for the file, saving this in the cache and than comparing this every time to "identify" the change of this file:
GET_PROPERTY(SCOPE GLOBAL PROPERTY  CACHED_IDL_CHECKSUM)  #Get saved checksum from previous run.
EXECUTE_PROCESS(COMMAND ${CMAKE_COMMAND}  -E md5sum ${IDL_FILE}
   OUTPUT_VARIABLE CalculatedCheckSum OUTPUT_STRIP_TRAILING_WHITESPACE) # generate new checksum
IF (NOT "${CACHED_IDL_CHECKSUM}"  STREQUAL  "${CalculatedCheckSum}") #Comapre checksums
   ....DO IDL Generation
    ... Update Checksum
ENDIF (NOT "${CACHED_IDL_CHECKSUM}"  STREQUAL  "${CalculatedCheckSum}")
 
 
So far so good.
But now, I need to take this a step further and add a custom rule so that I make sure that the above logic is execute on each and every build (i.e. reminder - the trigger to generate the IDL is IDL change and not CMakeLists.txt change).
It than that the "-P" comes into play, and I can try and do (where above logic is inside  IDL_COMPILE.cmake):
ADD_CUSTOM_TARGET(IDL_4 ${CMAKE_COMMAND} -P ${PROJECT_SOURCE_DIR}/IDL_COMPILE.cmake SOURCES "${PROJECT_SOURCE_DIR}/IDL_COMPILE.cmake")
ADD_DEPENDENCIES(${PROJECT_NAME} IDL_4)
 
I'm bumping into "solution killers" since, within the scope of "-P cmakescript" I need:
1. Use main CMAKE variables, i.e. PROJECT_NAME, CMAKE_COMMAND etc  (get's complicated as I need to use -D or CONFIGURE_FILE).
2. SET cache variables to update the checksum (Can't be done as per documentation).
3. Call Cmake Functions (Can't be done, I tried both functions in the -P file or from include() I have in my main script, both failed).
 
 
Any help on this will be much appreciated.
 
Regards,
        Avner Cohen.



----- Original Message ----
From: Eric Noulard <[hidden email]>
To: avner cohen <[hidden email]>
Cc: [hidden email]
Sent: Tuesday, August 11, 2009 11:23:02 AM
Subject: Re: [CMake] Checking Disk space with Cmake

2009/8/11 avner cohen <[hidden email]>:
>
> Thank you very much Eric, this defiently helps.
> My problem now is that I want my "<your_check_disk_command>" to be a MACRO/FUNCTION if already written in my cmake scripts (rather than running a shell script) is that possible ?

Almost yes, in fact you cannot "call a macro" in add_custom_target,
however you may put whatever you want in a checkdisk.cmake CMake script file
and then call this script  using CMake -P, like in:

add_custom_target(CheckDisk ${CMAKE_COMMAND} -P
${CMAKE_CURRENT_SOURCE_DIR}/checkdisk.cmake
                  COMMENT "Check available disk space before build"
                  SOURCES checkdisk.cmake)

If you need to pass value to your checkdisk.cmake, then you may use -D
arguments to CMake
and/or configure_file(checkdisk.cmake.in checkdisk.cmake)  in order to
make the called
script contains configured values from your project.


--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org



     
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Tyler Roscoe]

On Tue, Aug 11, 2009 at 11:28:14AM -0700, avner cohen wrote:
> But now, I need to take this a step further and add a custom rule so
> that I make sure that the above logic is execute on each and every
> build (i.e. reminder - the trigger to generate the IDL is IDL change
> and not CMakeLists.txt change).  It than that the "-P" comes into
> play, and I can try and do (where above logic is inside
>  IDL_COMPILE.cmake):

> ADD_CUSTOM_TARGET(IDL_4 ${CMAKE_COMMAND} -P ${PROJECT_SOURCE_DIR}/IDL_COMPILE.cmake SOURCES "${PROJECT_SOURCE_DIR}/IDL_COMPILE.cmake")
> ADD_DEPENDENCIES(${PROJECT_NAME} IDL_4)

Have you read and understood this?

http://www.cmake.org/Wiki/CMake_FAQ#How_can_I_generate_a_source_file_during_the_build.3F

That pattern should get you what you want.

tyler
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[avner cohen]

Thanks Tyler.

I'm not sure I'm following how this is related to my question.

The solution I'm looking for is to be able to execute or generate a file, ONLY when an out-of-project file is being modified (which is why I tried to use checksum caching). How is this condition satisifed in the below link?

Regards,
       Avner.



----- Original Message ----
From: Tyler Roscoe <[hidden email]>
To: avner cohen <[hidden email]>
Cc: Eric Noulard <[hidden email]>; [hidden email]
Sent: Tuesday, August 11, 2009 9:50:27 PM
Subject: Re: [CMake] Using Checksum to generate files

On Tue, Aug 11, 2009 at 11:28:14AM -0700, avner cohen wrote:
> But now, I need to take this a step further and add a custom rule so
> that I make sure that the above logic is execute on each and every
> build (i.e. reminder - the trigger to generate the IDL is IDL change
> and not CMakeLists.txt change).  It than that the "-P" comes into
> play, and I can try and do (where above logic is inside
>  IDL_COMPILE.cmake):

> ADD_CUSTOM_TARGET(IDL_4 ${CMAKE_COMMAND} -P ${PROJECT_SOURCE_DIR}/IDL_COMPILE.cmake SOURCES "${PROJECT_SOURCE_DIR}/IDL_COMPILE.cmake")
> ADD_DEPENDENCIES(${PROJECT_NAME} IDL_4)

Have you read and understood this?

http://www.cmake.org/Wiki/CMake_FAQ#How_can_I_generate_a_source_file_during_the_build.3F

That pattern should get you what you want.

tyler



     
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Eric Noulard]

2009/8/12 avner cohen <[hidden email]>:
>
> Thanks Tyler.
>
> I'm not sure I'm following how this is related to my question.
>
> The solution I'm looking for is to be able to execute or generate a file, ONLY when an out-of-project file is being modified (which is why I tried to use checksum caching). How is this condition satisifed in the below link?

As far as I understand your testcase I think both Tyler and Michael are right.
Why do you state that "when an out-of-project file is being modified"
your IDLs files are in the project right?

Your example says:
-I ${PROJECT_SOURCE_DIR}IDLs ${PROJECT_SOURCE_DIR}/IDLs/CORBAStructDefs.idl

so the COMPILE_IDL of Michael should be just fine, or are we missing something?

--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[avner cohen]

Eric,
I may have used wrong terminolegy there, my IDLs are indeed part of the the project, It's only that the project is a C++ based project, so the IDLs are not being "compiled" in any way, they are used to generated a set of .h and .cpp files (which in term, are patr of the project.
As soon as I generate these files, all compilers (xlC, cc and gcc) will execute a full build, even if the files are identical to the previously generated files.

So my main aim is to find a way to:
1. Always check if IDL changed.
2. Generated ouput .cpp and .h files only IDL are changed.

Thanks,

Avner.

 


----- Original Message ----
From: Eric Noulard <[hidden email]>
To: avner cohen <[hidden email]>
Cc: Tyler Roscoe <[hidden email]>; [hidden email]
Sent: Wednesday, August 12, 2009 10:26:48 AM
Subject: Re: [CMake] Using Checksum to generate files

2009/8/12 avner cohen <[hidden email]>:
>
> Thanks Tyler.
>
> I'm not sure I'm following how this is related to my question.
>
> The solution I'm looking for is to be able to execute or generate a file, ONLY when an out-of-project file is being modified (which is why I tried to use checksum caching). How is this condition satisifed in the below link?

As far as I understand your testcase I think both Tyler and Michael are right.
Why do you state that "when an out-of-project file is being modified"
your IDLs files are in the project right?

Your example says:
-I ${PROJECT_SOURCE_DIR}IDLs ${PROJECT_SOURCE_DIR}/IDLs/CORBAStructDefs.idl

so the COMPILE_IDL of Michael should be just fine, or are we missing something?

--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org



     
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Eric Noulard]

2009/8/12 avner cohen <[hidden email]>:
> Eric,
> I may have used wrong terminolegy there, my IDLs are indeed part of the the project, It's only that the project is a C++ based project, so the IDLs are not being "compiled" in any way, they are used to generated a set of .h and .cpp files (which in term, are patr of the project.
> As soon as I generate these files, all compilers (xlC, cc and gcc) will execute a full build, even if the files are identical to the previously generated files.

OK, this somehow a classical "generated source" topic.
Thus forget my first mails and look at what  Michael and Tyler gave you.

> So my main aim is to find a way to:
> 1. Always check if IDL changed.
> 2. Generated ouput .cpp and .h files only IDL are changed.

As far as I understand the CMake compile_idl function provided by Michael
does achieve that.

and the link
http://www.cmake.org/Wiki/CMake_FAQ#How_can_I_generate_a_source_file_during_the_build.3F
provided by Tyler gives the big picture on "generated source file".

--
Erk
Membre de l'April - « promouvoir et défendre le logiciel libre » -
http://www.april.org
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

[Michael Wild]

On 12. Aug, 2009, at 15:33, avner cohen wrote:

> Eric,
> I may have used wrong terminolegy there, my IDLs are indeed part of  
> the the project, It's only that the project is a C++ based project,  
> so the IDLs are not being "compiled" in any way, they are used to  
> generated a set of .h and .cpp files (which in term, are patr of the  
> project.
> As soon as I generate these files, all compilers (xlC, cc and gcc)  
> will execute a full build, even if the files are identical to the  
> previously generated files.
>
> So my main aim is to find a way to:
> 1. Always check if IDL changed.
> 2. Generated ouput .cpp and .h files only IDL are changed.
>
> Thanks,
>
> Avner.
>

Is it just me, or are you kind of impervious to advice? The  
add_custom_command in my example does EXACTLY what you want. It will  
always check whether the .idl file is newer than the corresponding .h  
and .cpp files, and regenerate the .h and .cpp files in that case by  
calling the IDL compiler.

Michael
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake