Visual Studio generator running custom_command twice

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Visual Studio generator running custom_command twice

Paul Smith
I have a situation where I've created a custom command to generate .cpp
files to be compiled (in my case running bison/flex).

I'm using CMake 3.13.4

However, I have to compile this code in two different ways for two
different targets.  I am seeing the Visual Studio generator re-running
bison/flex once for each of these targets, which is (a) not necessary
and (b) causes my builds to fail since it often happens that the second
run can't replace the output files (due to Windows' annoying habit of
locking files that are open).

I have a custom command like this:

 add_custom_command(OUTPUT
      ${OUT_DIR}/MyParser.tab.cpp
      ${OUT_DIR}/MyParser.tab.hpp
      COMMAND ${CMAKE_COMMAND} -E make_directory ${OUT_DIR}
      COMMAND ${BISON} -d -o ${OUT_DIR}/MyParser.tab.cpp MyParser.y
      DEPENDS MyParser.y
      COMMENT "Building MyParser parser")

  set(MyParserOutput
      ${OUT_DIR}/MyParser.tab.cpp
      ${OUT_DIR}/MyParser.tab.hpp)

  add_custom_target(MyGenParser DEPENDS ${MyParserOutput})

Then I have two different libraries, both depending on this:

  add_library(OneLib STATIC ${MyParserOutput} ...)

  target_compile_definitions(OneLib PRIVATE -DONELIB)

  target_include_directories(OneLib PRIVATE ${OUT_DIR})

  add_dependencies(OneLib MyGenparser)


  add_library(TwoLib STATIC ${MyParserOutput} ...)

  target_compile_definitions(TwoLib PRIVATE -DTWOLIB)

  target_include_directories(TwoLib PRIVATE ${OUT_DIR})

  add_dependencies(TwoLib MyGenparser)


On Linux and MacOS, this works fine.  However on Windows with Visual
Studio I see the custom target being run twice, once for each library:

  18:27:10 21>PrepareForBuild:
  18:27:10      Creating directory "OneLib.dir\RelWithDebInfo\".
  18:27:10      Creating directory "OneLib.dir\RelWithDebInfo\OneLib.tlog\".
  18:27:10    InitializeBuildStatus:
  18:27:10      Creating "OneLib.dir\RelWithDebInfo\OneLib.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
  18:27:10    ComputeCustomBuildOutput:
  18:27:10      Creating directory "D:\builds\src\".
  18:27:10    CustomBuild:
  18:27:10      Building MyParser parser

Then again later:

  18:27:20 69>PrepareForBuild:
  18:27:20      Creating directory "TwoLib.dir\RelWithDebInfo\".
  18:27:20      Creating directory "TwoLib.dir\RelWithDebInfo\TwoLib.tlog\".
  18:27:20    InitializeBuildStatus:
  18:27:20      Creating "TwoLib.dir\RelWithDebInfo\TwoLib.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
  18:27:20    CustomBuild:
  18:27:20      Building MyParser parser

See how this second one is re-building the parser; then this fails
because the output file is locked by Windows (in use by the compiler):

  18:27:21 69>CustomBuild:
  18:27:21      bison.exe: could not create D:/builds/src/MyParser.tab.cpp
  18:27:25 69>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(209,5): error MSB6006: "cmd.exe" exited with code 1. [D:\builds\TwoLib.vcxproj]

How can I get CMake to only run the bison command one time, then use
the output for two different libraries?

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Visual Studio generator running custom_command twice

frodak17


On Thu, Mar 14, 2019 at 1:13 AM Paul Smith <[hidden email]> wrote:
I have a situation where I've created a custom command to generate .cpp
files to be compiled (in my case running bison/flex).

I'm using CMake 3.13.4

  set(MyParserOutput
      ${OUT_DIR}/MyParser.tab.cpp
      ${OUT_DIR}/MyParser.tab.hpp)

  add_custom_target(MyGenParser DEPENDS ${MyParserOutput})

Then I have two different libraries, both depending on this:

  add_library(OneLib STATIC ${MyParserOutput} ...)
  add_dependencies(OneLib MyGenparser)


  add_library(TwoLib STATIC ${MyParserOutput} ...)
  add_dependencies(TwoLib MyGenparser)


From add_custom_command()
Do not list the output in more than one independent target that may build in parallel or the two instances of the rule may conflict (instead use the add_custom_target() command to drive the command and make the other targets depend on that one)

Removing ${MyParserOutput} from both add_library() should fix the issue.

Best regards,
F

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Visual Studio generator running custom_command twice

frodak17


On Thu, Mar 14, 2019 at 12:53 PM frodak17 <[hidden email]> wrote:


On Thu, Mar 14, 2019 at 1:13 AM Paul Smith <[hidden email]> wrote:
I have a situation where I've created a custom command to generate .cpp
files to be compiled (in my case running bison/flex).

I'm using CMake 3.13.4

  set(MyParserOutput
      ${OUT_DIR}/MyParser.tab.cpp
      ${OUT_DIR}/MyParser.tab.hpp)

  add_custom_target(MyGenParser DEPENDS ${MyParserOutput})

Then I have two different libraries, both depending on this:

  add_library(OneLib STATIC ${MyParserOutput} ...)
  add_dependencies(OneLib MyGenparser)


  add_library(TwoLib STATIC ${MyParserOutput} ...)
  add_dependencies(TwoLib MyGenparser)


From add_custom_command()
Do not list the output in more than one independent target that may build in parallel or the two instances of the rule may conflict (instead use the add_custom_target() command to drive the command and make the other targets depend on that one)

Removing ${MyParserOutput} from both add_library() should fix the issue.

Best regards,
F

Sorry, It didn't register at first that about the cpp file that needs to get compiled into the library.

That makes it a little more complicated.

In that case you need to keep ${MyParserOutput} and set the GENERATED properties for the files.

Also the building the custom target needs to be done in a separate directory as the add_custom_commands() need to be in a different CMakeLists.txt file from the libraries.  Otherwise the rules get pulled into the libraries and cause the commands to be run multiple times.




--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Visual Studio generator running custom_command twice

Paul Smith
On Thu, 2019-03-14 at 13:30 -0400, frodak17 wrote:
On Thu, Mar 14, 2019 at 12:53 PM frodak17 <[hidden email]> wrote:
On Thu, Mar 14, 2019 at 1:13 AM Paul Smith <[hidden email]> wrote:
I have a situation where I've created a custom command to generate .cpp
files to be compiled (in my case running bison/flex).

I'm using CMake 3.13.4

  set(MyParserOutput
      ${OUT_DIR}/MyParser.tab.cpp
      ${OUT_DIR}/MyParser.tab.hpp)

  add_custom_target(MyGenParser DEPENDS ${MyParserOutput})

Then I have two different libraries, both depending on this:

  add_library(OneLib STATIC ${MyParserOutput} ...)
  add_dependencies(OneLib MyGenparser)


  add_library(TwoLib STATIC ${MyParserOutput} ...)
  add_dependencies(TwoLib MyGenparser)

From add_custom_command()
Do not list the output in more than one independent target that may build in parallel or the two instances of the rule may conflict (instead use the add_custom_target() command to drive the command and make the other targets depend on that one)

Yeah, I did see that in the manual and I do that, as above. It wan't at all clear to me that in addition to those requirements, I ALSO could NOT list the output of the add_custom_command() in any of my other targets.

That's unfortunate :(.

In that case you need to keep ${MyParserOutput} and set the GENERATED properties for the files.

Also the building the custom target needs to be done in a separate directory as the add_custom_commands() need to be in a different CMakeLists.txt file from the libraries.  Otherwise the rules get pulled into the libraries and cause the commands to be run multiple times.

Ouch. That's painful as it could mean moving my code around.

However in this case it ended up not being too horrible because my parser input files happened to be in a subdirectory already, so I created a new CMakeLists.txt file there that only ran the generators, then used set(... PARENT_SCOPE) to publish the variables to the parent directory.

I did then have to add a GENERATED property to those files in the parent directory, as you noted, so overall it's a bit leaky in terms of abstractions, but it seems to work... now when I run on Windows I see that the generators are run only one time.

Thanks for the help!!

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake