how to force assign sequence in multi-thread in cmake

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

how to force assign sequence in multi-thread in cmake

Yu Jing

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake ..
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Petr Kmoch
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake ..
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Micha Hergarden
Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake ..
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake





--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Yu Jing
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake ..
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake






--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Micha Hergarden
It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake ..
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake







--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Yu Jing
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake ..
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake








--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Marcel Loose
Hi Yu,

I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.

HTH,
Marcel Loose.


On 16/09/14 10:41, Yu Jing wrote:
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake ..<
/span>
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake .
.
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake











--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

loose.vcf (304 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Micha Hergarden
Hello Yu,

It seems I too needed another trick to make it work:
set_property(TARGET CRFPP_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
set_property(TARGET LEVELDB_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)

ADD_DEPENDENCIES(cmake_sample iniparser_static CRFPP_EX_PROJ LEVELDB_EX_PROJ)

By default the ExternalProject_Add will add that target to the ALL target, and that confuses the Make dependency checker. I have removed those dependencies in my project. On the whole I agree with Marcel Loose and Petr Kmoch however. Combining their solution makes for a 'cleaner' project. To be honest, I am halfway of migrating my project to a superbuild as well. Take a look at the openchemistry project if you would like to see an example of that: https://github.com/OpenChemistry/openchemistry

Regards,
Micha Hergarden


On 09/16/2014 11:04 AM, Marcel Loose wrote:
Hi Yu,

I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.

HTH,
Marcel Loose.


On 16/09/14 10:41, Yu Jing wrote:
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake .
.
 <
/span>
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake 
 .
.
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake














--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Yu Jing
Hello Micha,
It seems still not work….

On Sep 16, 2014, at 17:36, Micha Hergarden <[hidden email]> wrote:

Hello Yu,

It seems I too needed another trick to make it work:
set_property(TARGET CRFPP_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
set_property(TARGET LEVELDB_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)

ADD_DEPENDENCIES(cmake_sample iniparser_static CRFPP_EX_PROJ LEVELDB_EX_PROJ)

By default the ExternalProject_Add will add that target to the ALL target, and that confuses the Make dependency checker. I have removed those dependencies in my project. On the whole I agree with Marcel Loose and Petr Kmoch however. Combining their solution makes for a 'cleaner' project. To be honest, I am halfway of migrating my project to a superbuild as well. Take a look at the openchemistry project if you would like to see an example of that: https://github.com/OpenChemistry/openchemistry

Regards,
Micha Hergarden


On 09/16/2014 11:04 AM, Marcel Loose wrote:
Hi Yu,

I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.

HTH,
Marcel Loose.


On 16/09/14 10:41, Yu Jing wrote:
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake .
.
 <
/span>
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake 
 .
.
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake













--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Micha Hergarden
Hello Yu,

That is because of the following lines:

AUX_SOURCE_DIRECTORY(. DIR_TARGET)
ADD_LIBRARY (relfiles ${DIR_TARGET})

I think you may be misusing the aux_source_directory command. I have tested your setup on a linux machine with -j8 and it still builds. Did you take a look at the openchemistry setup?

Regards,
Micha

On 09/16/2014 01:42 PM, Yu Jing wrote:
Hello Micha,
It seems still not work….

On Sep 16, 2014, at 17:36, Micha Hergarden <[hidden email]> wrote:

Hello Yu,

It seems I too needed another trick to make it work:
set_property(TARGET CRFPP_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
set_property(TARGET LEVELDB_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)

ADD_DEPENDENCIES(cmake_sample iniparser_static CRFPP_EX_PROJ LEVELDB_EX_PROJ)

By default the ExternalProject_Add will add that target to the ALL target, and that confuses the Make dependency checker. I have removed those dependencies in my project. On the whole I agree with Marcel Loose and Petr Kmoch however. Combining their solution makes for a 'cleaner' project. To be honest, I am halfway of migrating my project to a superbuild as well. Take a look at the openchemistry project if you would like to see an example of that: https://github.com/OpenChemistry/openchemistry

Regards,
Micha Hergarden


On 09/16/2014 11:04 AM, Marcel Loose wrote:
Hi Yu,

I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.

HTH,
Marcel Loose.


On 16/09/14 10:41, Yu Jing wrote:
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake .
.
 <
/span>
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake 
 .
.
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake













--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Yu Jing
I am trying … That’s a very clear organization 


On Sep 16, 2014, at 19:57, Micha Hergarden <[hidden email]> wrote:

Hello Yu,

That is because of the following lines:

AUX_SOURCE_DIRECTORY(. DIR_TARGET)
ADD_LIBRARY (relfiles ${DIR_TARGET})

I think you may be misusing the aux_source_directory command. I have tested your setup on a linux machine with -j8 and it still builds. Did you take a look at the openchemistry setup?

Regards,
Micha

On 09/16/2014 01:42 PM, Yu Jing wrote:
Hello Micha,
It seems still not work….

On Sep 16, 2014, at 17:36, Micha Hergarden <[hidden email]> wrote:

Hello Yu,

It seems I too needed another trick to make it work:
set_property(TARGET CRFPP_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
set_property(TARGET LEVELDB_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)

ADD_DEPENDENCIES(cmake_sample iniparser_static CRFPP_EX_PROJ LEVELDB_EX_PROJ)

By default the ExternalProject_Add will add that target to the ALL target, and that confuses the Make dependency checker. I have removed those dependencies in my project. On the whole I agree with Marcel Loose and Petr Kmoch however. Combining their solution makes for a 'cleaner' project. To be honest, I am halfway of migrating my project to a superbuild as well. Take a look at the openchemistry project if you would like to see an example of that: https://github.com/OpenChemistry/openchemistry

Regards,
Micha Hergarden


On 09/16/2014 11:04 AM, Marcel Loose wrote:
Hi Yu,

I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.

HTH,
Marcel Loose.


On 16/09/14 10:41, Yu Jing wrote:
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake .
.
 <
/span>
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake 
 .
.
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake













--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake




--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: how to force assign sequence in multi-thread in cmake

Yu Jing
Hello All,
It is is because of 
# AUX_SOURCE_DIRECTORY(. DIR_TARGET)
# ADD_LIBRARY (relfiles ${DIR_TARGET})

        That’s really a stupid fault ,and thanks for your debug …TAT

        I adjust the usage of External project , and it seems much better . Thanks for your help . Thanks for Micha Hergarden’s patience …

        By the way , 
set_property(TARGET leveldb_proj PROPERTY EXCLUDE_FROM_ALL TRUE) 
         I removed this property and it seems still works.



On Sep 16, 2014, at 19:58, Yu Jing <[hidden email]> wrote:

I am trying … That’s a very clear organization 


On Sep 16, 2014, at 19:57, Micha Hergarden <[hidden email]> wrote:

Hello Yu,

That is because of the following lines:

AUX_SOURCE_DIRECTORY(. DIR_TARGET)
ADD_LIBRARY (relfiles ${DIR_TARGET})

I think you may be misusing the aux_source_directory command. I have tested your setup on a linux machine with -j8 and it still builds. Did you take a look at the openchemistry setup?

Regards,
Micha

On 09/16/2014 01:42 PM, Yu Jing wrote:
Hello Micha,
It seems still not work….

On Sep 16, 2014, at 17:36, Micha Hergarden <[hidden email]> wrote:

Hello Yu,

It seems I too needed another trick to make it work:
set_property(TARGET CRFPP_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)
set_property(TARGET LEVELDB_EX_PROJ PROPERTY EXCLUDE_FROM_ALL TRUE)

ADD_DEPENDENCIES(cmake_sample iniparser_static CRFPP_EX_PROJ LEVELDB_EX_PROJ)

By default the ExternalProject_Add will add that target to the ALL target, and that confuses the Make dependency checker. I have removed those dependencies in my project. On the whole I agree with Marcel Loose and Petr Kmoch however. Combining their solution makes for a 'cleaner' project. To be honest, I am halfway of migrating my project to a superbuild as well. Take a look at the openchemistry project if you would like to see an example of that: https://github.com/OpenChemistry/openchemistry

Regards,
Micha Hergarden


On 09/16/2014 11:04 AM, Marcel Loose wrote:
Hi Yu,

I think you need to add an explicit dependency of main.cc on the "generated" (well, not really generated, but installed) header file crfpp.h. CMake has no clue as to what files are being compiled/installed by your external project, so you have to make this explicit.

HTH,
Marcel Loose.


On 16/09/14 10:41, Yu Jing wrote:
Hello  Micha ,

It seems doesn’t work.
I updated code , and move the external project to src , and it still not work.

what should be noticed is :
1. If I just use 
$ make
as make command ,all styles are work
2. If I use in multi jobs
$ make -j8
It will run abnormal (not desired sequence) .


On Sep 16, 2014, at 15:27, Micha Hergarden <[hidden email]> wrote:

It may be that you have line 63 and 64 the wrong way around:
ADD_SUBDIRECTORY(src)
ADD_SUBDIRECTORY(lib)

The externalproject is added in lib, but you add a dependency on it in src. CMake will descend in the subdirectories in the order you supply them.
Does reversing the directories help?

Regards,
Micha

On 09/16/2014 09:17 AM, Yu Jing wrote:
I am in OSX 10.9.4 , a sample in  github is : https://github.com/yujing5b5d/cmake_sample
after git clone this project , a operation like this :
------------------------------------------------------------------------------------------------------------

yu:cmake_sample yu$ mkdir build
yu:cmake_sample yu$ cd build/
yu:build yu$ cmake ..
…. # skip some useless output
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/yu/Workspace/res/cmake_sample/build
yu:build yu$ make -j8 ### <<<<<<<<<<<<<<<
Scanning dependencies of target LEVELDB_EX_PROJ
Scanning dependencies of target iniparser
Scanning dependencies of target relfiles
Scanning dependencies of target CRFPP_EX_PROJ
Scanning dependencies of target iniparser_static
[ 15%] [ 15%] [ 20%] [ 20%] [ 25%] Building C object lib/iniparser/CMakeFiles/iniparser_static.dir/ini.c.o
Creating directories for 'CRFPP_EX_PROJ'
Creating directories for 'LEVELDB_EX_PROJ'
Building C object lib/iniparser/CMakeFiles/iniparser.dir/ini.c.o
Building CXX object src/CMakeFiles/relfiles.dir/main.cc.o
[ 30%] [ 35%] Performing download step (git clone) for 'LEVELDB_EX_PROJ'
Performing download step (git clone) for 'CRFPP_EX_PROJ'
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
Cloning into 'CRFPP_EX_PROJ'...
Cloning into 'LEVELDB_EX_PROJ'...
Linking C static library ../libiniparser.a
Linking C shared library ../libiniparser.dylib
[ 35%] [ 35%] Built target iniparser_static
Built target iniparser
Scanning dependencies of target cmake_sample
[ 40%] Building CXX object src/CMakeFiles/cmake_sample.dir/main.cc.o
/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/relfiles.dir/main.cc.o] Error 1
make[2]: *** [src/CMakeFiles/cmake_sample.dir/main.cc.o] Error 1
make[1]: *** [src/CMakeFiles/relfiles.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/CMakeFiles/cmake_sample.dir/all] Error 2
….
------------------------------------------------------------------------------------------------------------

BE CAREFUL OF THIS LINE :
>> yu:build yu$ make -j8

the ExternalProject CRFPP_EX_PROJ’s result contains copy a crfpp.h  header to a special path, after this process , we can use #include “crfpp.h" ,and If I use 
make -j8
this means 8 jobs can be running at the same time, I can not constraints and let my compiler compile my main.cc after CRFPP_EX_PROJ finished.

Of course , I’m not sure is this my misuse this project .



On Sep 16, 2014, at 14:55, Micha Hergarden <[hidden email]> wrote:

Hello all,

I do use the ExternalProject to prebuild some binaries, without the 'superproject' setup, and it does seem to work. Using the add_dependencies, I can make sure some third party libs are prebuild before I start to build my project. I have seen some issues with ExternalProject (failing to extract, or build), but they are too rare to pinpoint and create a bugreport.

What exactly does not work? Is the external project not build at all, or just not in time?

Regards,
Micha

On 09/16/2014 08:30 AM, Petr Kmoch wrote:
Hi.

I've never worked with ExternalProject myself, so I can't comment with certainty, but from what I understand, the correct way of using ExternalProject is to add your own project as an ExternalProject as well. Basically, the toplevel CMakeList becomes a superbuild which *only* does ExternalProject_Add() calls and does not add any libraries/executables directly. After you build the superbuild once to get all the dependencies correct, you switch to the "external" project of your own code and work with that normally.

Petr

On Mon, Sep 15, 2014 at 4:13 PM, Yu Jing <[hidden email]> wrote:

I had asked in http://stackoverflow.com/questions/25841602  , someone told me maybe here is a better place to ask.

I am writing a project base on crfpp , a external project. I use cmake to integerate this project as follow .

firstly , I add a extenal project like this:

EXTERNALPROJECT_ADD(
            CRFPP_EX_PROJ
            GIT_REPOSITORY git@github.com:yujing5b5d/crfpp.git
            PREFIX ${CMAKE_CURRENT_BINARY_DIR}
            CONFIGURE_COMMAND ./configure
            BUILD_COMMAND make -j8
            BUILD_IN_SOURCE 1
            INSTALL_COMMAND cp .libs/libcrfpp.a ${PROJECT_BINARY_DIR}/lib && cp crfpp.h ${PROJECT_BINARY_DIR}/include 
    )

this will generate some .a file and copy a header file crfpp.h to folder ${PROJECT_BINARY_DIR}/include , which is included in my project.

and then , use the ${PROJECT_BINARY_DIR}/include as include path as follow .

INCLUDE_DIRECTORIES(
    ${PROJECT_SOURCE_DIR}/include
    ${PROJECT_BINARY_DIR}/include
)

finally , when I compile the main project , code like this :

ADD_EXECUTABLE(cmake_sample main.cc)
ADD_DEPENDENCIES(cmake_sample CRFPP_EX_PROJ)
TARGET_LINK_LIBRARIES(cmake_sample crfpp)

In general , if I just build a build folder, compile like this :

cmake .
.
 <
/span>
make

It may works fine, compile the external project first , and copy the header file to desired place , and then continue compile main.cc in my project . But if I use compile command as multi-thread like this :

cmake 
 .
.
make -j8

It will not works because my main.cc and the external project are processed at same time, so it report a error like this :

/Users/yu/Workspace/res/cmake_sample/src/main.cc:3:10: fatal error: 'crfpp.h' file not found
#include "crfpp.h" // crfpp
         ^
1 error generated.

This crfpp.h will generated after CRFPP_EX_PROJ , but in multi-thread environment , the sequence is quite different .

My Question is : Is it possible that force let my project compile after these external projects all finished . BE CAREFUL , I'm not sure is this the problem of my use of ADD_DEPENDENCIES , I also wrote

ADD_DEPENDENCIES(
<
span style="margin:0px;padding:0px;border:0px;vertical-align:baseline;background-color:transparent;background-repeat:initial initial">cmake_sample CRFPP_EX_PROJ)

but it seems not works ?

Thanks for any help.



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake













--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake





--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake