Embedded build using ExternalProject

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Embedded build using ExternalProject

Torsten Robitzki
Hello,

I have a project, where binaries are build for an embedded platform and for a PC platform. To build software update packages for the embedded platforms (files that can be used to make in the field firmware updates), tools that have to run during the build have to be build from source first. For example firmware.tfp have to be build by running create_package with firmware.bin. firmware.bin have to build with settings for the embedded platform and create_package have to be build with setting of the build host.

I’ve asked this question already a year or so ago and got the response, that using super builds will help me. Now I try to convert the project to a super build structure. Fortunately, there was very little content inside my main CMakeLists.txt, mainly `add_subdirectory( source )`. Mainly, I want to build different sets of artifacts from the same source tree, but with different

Now I’m stuck with a number of issues. First, I want to build all firmware binaries for the NRF52 embedded platform:

    ExternalProject_Add(firmware_binaries
        SOURCE_DIR  ${CMAKE_CURRENT_LIST_DIR}/source
        INSTALL_DIR ${installDir}
        CMAKE_ARGS
            -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
            -DBOOTLOADER_AES_KEY=${BOOTLOADER_AES_KEY}
            -DBOOTLOADER_VERSION=${BOOTLOADER_VERSION}
            -DBUILD_DEBUG=1
            -DBUILD_NRF52=1
        BUILD_COMMAND ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR} --target firmware_binaries
    )

I don’t want to build any EOL test firmware, test-tools etc. just the stuff that will be shipped, thus the explicit target. I can build the firmware_binaries, from the source directly, without problems. But when I try to build with the top-level cmake file, that contains the ExternalProject_Add(), I get the usual configure output and then:

-- Configuring done
-- Generating done
-- Build files have been written to: /Users/todi/blueswd_firmware/build/firmware_binaries-prefix/src/firmware_binaries-build
[ 75%] Performing build step for 'firmware_binaries'
[ 12%] Performing build step for 'firmware_binaries'
[ 12%] Performing build step for 'firmware_binaries'
[ 12%] Performing build step for ‚firmware_binaries‘


This line repeats until make reports:

make[776]: /usr/local/Cellar/cmake/3.14.3/bin/cmake: Resource temporarily unavailable
make[776]: *** wait: No child processes.  Stop.
make[776]: *** Waiting for unfinished jobs....
make[776]: *** wait: No child processes.  Stop.

Any idea, what might be wrong here? Some kind of recursion?

An other issue: I want to forward targets to the super build, so that I can build them there. For example I have ~30 unit test programs and when I’m working in a certain area, I usually want to build exactly one of this programs. Is it possible to export them as targets of the super build?

A third issue that I have is, that changes to the CMakeLists.txt files in the Project are not reflected by the super build. If I make a change to one of the CMake files under ${CMAKE_CURRENT_LIST_DIR}/source, I usually have to delete the whole build folder and start from scratch with the build. Is there an other way to force the super build to recognize the changes? (Note: ${CMAKE_CURRENT_LIST_DIR}/source is not added as subdirectory to the top level (super build) CMakeLists.txt).

best regards,

Torsten

P.S.:
$ cmake --version
cmake version 3.14.3


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Embedded build using ExternalProject

Albrecht Schlosser
On 6/26/19 3:53 PM Torsten Robitzki wrote:

> I don’t want to build any EOL test firmware, test-tools etc. just the stuff that will be shipped, thus the explicit target. I can build the firmware_binaries, from the source directly, without problems. But when I try to build with the top-level cmake file, that contains the ExternalProject_Add(), I get the usual configure output and then:
>
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /Users/todi/blueswd_firmware/build/firmware_binaries-prefix/src/firmware_binaries-build

Is this a on a Windows system?

Are you maybe using Cygwin or MinGW?

> [ 75%] Performing build step for 'firmware_binaries'
> [ 12%] Performing build step for 'firmware_binaries'
> [ 12%] Performing build step for 'firmware_binaries'
> [ 12%] Performing build step for ‚firmware_binaries‘
> …
>
> This line repeats until make reports:
>
> make[776]: /usr/local/Cellar/cmake/3.14.3/bin/cmake: Resource temporarily unavailable
> make[776]: *** wait: No child processes.  Stop.
> make[776]: *** Waiting for unfinished jobs....
> make[776]: *** wait: No child processes.  Stop.
>
> Any idea, what might be wrong here? Some kind of recursion?

"Resource temporarily unavailable" looks like a well-known Cygwin (or
maybe also MinGW) issue. If both of your answers above is "yes" I'd
guess it is.

As a workaround you may try to use 'make -j1' or something like this to
reduce the number of concurrent make jobs, but this is only a guess.
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedded build using ExternalProject

Torsten Robitzki
Hi Albrecht,

> Am 26.06.2019 um 19:10 schrieb Albrecht Schlosser <[hidden email]>:
>
> On 6/26/19 3:53 PM Torsten Robitzki wrote:
>
>> I don’t want to build any EOL test firmware, test-tools etc. just the stuff that will be shipped, thus the explicit target. I can build the firmware_binaries, from the source directly, without problems. But when I try to build with the top-level cmake file, that contains the ExternalProject_Add(), I get the usual configure output and then:
>> -- Configuring done
>> -- Generating done
>> -- Build files have been written to: /Users/todi/blueswd_firmware/build/firmware_binaries-prefix/src/firmware_binaries-build
>
> Is this a on a Windows system?
No, my build host is a Mac. I figured out, what the problem with this recusion is:

  BUILD_COMMAND ${CMAKE_COMMAND} —build ${CMAKE_CURRENT_BINARY_DIR} --target firmware_binaries

The build command really recuses into ${CMAKE_CURRENT_BINARY_DIR}, which have to be:

  BUILD_COMMAND ${CMAKE_COMMAND} —build <BINARY_DIR> --target firmware_packages

:-)






--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Embedded build using ExternalProject

Hendrik Sattler
In reply to this post by Torsten Robitzki


Am 26. Juni 2019 15:53:16 MESZ schrieb Torsten Robitzki <[hidden email]>:

>Hello,
>
>I have a project, where binaries are build for an embedded platform and
>for a PC platform. To build software update packages for the embedded
>platforms (files that can be used to make in the field firmware
>updates), tools that have to run during the build have to be build from
>source first. For example firmware.tfp have to be build by running
>create_package with firmware.bin. firmware.bin have to build with
>settings for the embedded platform and create_package have to be build
>with setting of the build host.
>
>I’ve asked this question already a year or so ago and got the response,
>that using super builds will help me. Now I try to convert the project
>to a super build structure. Fortunately, there was very little content
>inside my main CMakeLists.txt, mainly `add_subdirectory( source )`.
>Mainly, I want to build different sets of artifacts from the same
>source tree, but with different
>
>Now I’m stuck with a number of issues. First, I want to build all
>firmware binaries for the NRF52 embedded platform:
>
>    ExternalProject_Add(firmware_binaries
>        SOURCE_DIR  ${CMAKE_CURRENT_LIST_DIR}/source
>        INSTALL_DIR ${installDir}
>        CMAKE_ARGS
>            -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
>            -DBOOTLOADER_AES_KEY=${BOOTLOADER_AES_KEY}
>            -DBOOTLOADER_VERSION=${BOOTLOADER_VERSION}
>            -DBUILD_DEBUG=1
>            -DBUILD_NRF52=1
>BUILD_COMMAND ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR}
>--target firmware_binaries
>    )

The build command is using the wrong directory.

>I don’t want to build any EOL test firmware, test-tools etc. just the
>stuff that will be shipped, thus the explicit target. I can build the
>firmware_binaries, from the source directly, without problems. But when
>I try to build with the top-level cmake file, that contains the
>ExternalProject_Add(), I get the usual configure output and then:
>
>-- Configuring done
>-- Generating done
>-- Build files have been written to:
>/Users/todi/blueswd_firmware/build/firmware_binaries-prefix/src/firmware_binaries-build
>[ 75%] Performing build step for 'firmware_binaries'
>[ 12%] Performing build step for 'firmware_binaries'
>[ 12%] Performing build step for 'firmware_binaries'
>[ 12%] Performing build step for ‚firmware_binaries‘
>…
>
>This line repeats until make reports:
>
>make[776]: /usr/local/Cellar/cmake/3.14.3/bin/cmake: Resource
>temporarily unavailable
>make[776]: *** wait: No child processes.  Stop.
>make[776]: *** Waiting for unfinished jobs....
>make[776]: *** wait: No child processes.  Stop.
>
>Any idea, what might be wrong here? Some kind of recursion?
>
>An other issue: I want to forward targets to the super build, so that I
>can build them there. For example I have ~30 unit test programs and
>when I’m working in a certain area, I usually want to build exactly one
>of this programs. Is it possible to export them as targets of the super
>build?
>
>A third issue that I have is, that changes to the CMakeLists.txt files
>in the Project are not reflected by the super build. If I make a change
>to one of the CMake files under ${CMAKE_CURRENT_LIST_DIR}/source, I
>usually have to delete the whole build folder and start from scratch
>with the build. Is there an other way to force the super build to
>recognize the changes? (Note: ${CMAKE_CURRENT_LIST_DIR}/source is not
>added as subdirectory to the top level (super build) CMakeLists.txt).
>
>best regards,
>
>Torsten
>
>P.S.:
>$ cmake --version
>cmake version 3.14.3
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedded build using ExternalProject

Torsten Robitzki


> Am 26.06.2019 um 20:07 schrieb Hendrik Sattler <[hidden email]>:
>
>>   ExternalProject_Add(firmware_binaries
>>       SOURCE_DIR  ${CMAKE_CURRENT_LIST_DIR}/source
>>       INSTALL_DIR ${installDir}
>>       CMAKE_ARGS
>>           -DCMAKE_INSTALL_PREFIX:PATH=<INSTALL_DIR>
>>           -DBOOTLOADER_AES_KEY=${BOOTLOADER_AES_KEY}
>>           -DBOOTLOADER_VERSION=${BOOTLOADER_VERSION}
>>           -DBUILD_DEBUG=1
>>           -DBUILD_NRF52=1
>> BUILD_COMMAND ${CMAKE_COMMAND} --build ${CMAKE_CURRENT_BINARY_DIR}
>> --target firmware_binaries
>>   )
>
> The build command is using the wrong directory.
Yes, thank you.

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Embedded build using ExternalProject

Eric Dönges
In reply to this post by Torsten Robitzki


Am 26.06.19 um 15:53 schrieb Torsten Robitzki:
A third issue that I have is, that changes to the CMakeLists.txt files in the Project are not reflected by the super build. If I make a change to one of the CMake files under ${CMAKE_CURRENT_LIST_DIR}/source, I usually have to delete the whole build folder and start from scratch with the build. Is there an other way to force the super build to recognize the changes? (Note: ${CMAKE_CURRENT_LIST_DIR}/source is not added as subdirectory to the top level (super build) CMakeLists.txt).

I can't help you with your other issues, but I believe the solution to this problem is to specify "BUILD_ALWAYS TRUE" in your ExternalProject_Add. From the documentation for ExternalProject_Add:

Enabling this option forces the build step to always be run. This can be the easiest way to robustly ensure that the external project’s own build dependencies are evaluated rather than relying on the default success timestamp-based method. This option is not normally needed unless developers are expected to modify something the external project’s build depends on in a way that is not detectable via the step target dependencies (e.g. SOURCE_DIR is used without a download method and developers might modify the sources in SOURCE_DIR).

--

Dr. Eric Dönges
Senior Software Engineer

MVTec Software GmbH | Arnulfstr. 205 | 80634 Munich | Germany
[hidden email] | Tel: +49 89 457 695-0 | www.mvtec.com

 Sign up for our MVTec Newsletter!

Geschäftsführer: Dr. Wolfgang Eckstein, Dr. Olaf Munkelt
Amtsgericht München HRB 114695

 

MVTec Software GmbH Logo

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Embedded build using ExternalProject

Torsten Robitzki
Hi Eric,

> Am 27.06.2019 um 13:06 schrieb Eric Doenges <[hidden email]>:
>
> I can't help you with your other issues, but I believe the solution to this problem is to specify "BUILD_ALWAYS TRUE" in your ExternalProject_Add. From the documentation for ExternalProject_Add:
> Enabling this option forces the build step to always be run. This can be the easiest way to robustly ensure that the external project’s own build dependencies are evaluated rather than relying on the default success timestamp-based method. This option is not normally needed unless developers are expected to modify something the external project’s build depends on in a way that is not detectable via the step target dependencies (e.g. SOURCE_DIR is used without a download method and developers might modify the sources in SOURCE_DIR).

I’ve applied this option and will see if it helps.

thanks,

Torsten

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

signature.asc (849 bytes) Download Attachment