Generator expressions: Identifying when they're used in a custom command?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Generator expressions: Identifying when they're used in a custom command?

Sam Edwards
Hi list!

I'm working on a CMake buildsystem for the Panda3D project. The project is essentially a C++ 3D engine with its own custom Python binding generator known as Interrogate. The basic idea is that after a library's C++ code is compiled in the usual way, the C++ source files and headers are passed to Interrogate which parses a (very large) subset of the C++ language and generates suitable C++ to bind the library to the CPython API. This is then compiled into a CPython extension module and linked against the C++ code.

Because Interrogate is invoked in much the same way as a compiler, and as such needs to know the include search path, I use a generator expression to read the INTERFACE_INCLUDE_DIRECTORIES target property for the library in question and produce -I flags for Interrogate. So far, so good.

But again, Interrogate only knows a subset of C++, and while the Panda3D core is written with this in mind, this is a problem for third-party header files which might go outside of this subset. To solve this, the Panda3D source repository contains a "parser-inc" directory which contains stubbed-out header files solely intended for Interrogate. This means *hiding* the third-party package paths from Interrogate when it walks over INTERFACE_INCLUDE_DIRECTORIES.

"No problem," I thought, "generator expressions do exactly that." So I wrote a generator expression that looks at the highest target and, if it has the IS_INTERROGATE property, hides it.

However, when it came time to actually set this IS_INTERROGATE property, I could find no way to apply it to a custom command. I guess this makes sense - custom commands aren't targets, after all. But I couldn't convert my custom command to a custom target, because it produces a C++ source file and I need to use OUTPUT to let CMake know where that source file comes from, and you can't use OUTPUT and TARGET together in add_custom_command.

At the end of the day, I ended up using this as my workaround. It's really quite awful, and I'm not happy at all about the idea of maintaining it, but it does get the job done. Thus, my two questions are:
1) Is there a less messy way of accomplishing what I'm trying to do (that works in CMake 2.8.12+)? and
2) If not, should CMake be given an informational generator expression going forward that can determine context when targets aren't involved?

Or put another way, "Am I missing something, or is this actually a shortcoming in CMake?"

Thank you for your time, and I hope you're having a good day,
Sam

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generator expressions: Identifying when they're used in a custom command?

Alan W. Irwin
On 2018-02-18 03:49-0700 Sam Edwards wrote:

> However, when it came time to actually set this IS_INTERROGATE property, I
> could find no way to apply it to a custom command. I guess this makes sense
> - custom commands aren't targets, after all. But I couldn't convert my
> custom command to a custom target, because it produces a C++ source file
> and I need to use OUTPUT to let CMake know where that source file comes
> from, and you can't use OUTPUT and TARGET together in add_custom_command.

Hi Sam:

It's quite common for each custom command to have a corresponding
custom target that DEPENDS on the OUTPUT of the custom command.  So
building the target part of that pair means the custom command is
executed *only if* the OUTPUT from it is out of date.

If you used that paradigm for the custom commands you refer to
above (where each such custom command is paired with a unique target
via the DEPENDS of the latter) could you not set the IS_INTERROGATE
property for the custom target part of of each pair whenever that
property is relevant?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generator expressions: Identifying when they're used in a custom command?

Sam Edwards
Alan,

Thanks for your help! I tried to implement that paradigm myself in a small example CMakeLists.txt which I've attached, but the addition of a depending custom target doesn't seem to change the context of the 'this' target. In other words, the IS_INTERROGATE property is still read from fake_target and not depending_target.

Could you take a look and clarify what I'm not doing quite right?

Best,
Sam

On Sun, Feb 18, 2018 at 3:09 PM, Alan W. Irwin <[hidden email]> wrote:
On 2018-02-18 03:49-0700 Sam Edwards wrote:

However, when it came time to actually set this IS_INTERROGATE property, I
could find no way to apply it to a custom command. I guess this makes sense
- custom commands aren't targets, after all. But I couldn't convert my
custom command to a custom target, because it produces a C++ source file
and I need to use OUTPUT to let CMake know where that source file comes
from, and you can't use OUTPUT and TARGET together in add_custom_command.

Hi Sam:

It's quite common for each custom command to have a corresponding
custom target that DEPENDS on the OUTPUT of the custom command.  So
building the target part of that pair means the custom command is
executed *only if* the OUTPUT from it is out of date.

If you used that paradigm for the custom commands you refer to
above (where each such custom command is paired with a unique target
via the DEPENDS of the latter) could you not set the IS_INTERROGATE
property for the custom target part of of each pair whenever that
property is relevant?

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

CMakeLists.txt (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Generator expressions: Identifying when they're used in a custom command?

Alan W. Irwin
On 2018-02-18 19:47-0700 Sam Edwards wrote:

> Alan,
>
> Thanks for your help! I tried to implement that paradigm myself in a small
> example CMakeLists.txt which I've attached, but the addition of a depending
> custom target doesn't seem to change the context of the 'this' target. In
> other words, the IS_INTERROGATE property is still read from fake_target and
> not depending_target.
>
> Could you take a look and clarify what I'm not doing quite right?

Hi Sam:

I just looked at the custom command part of it which was

add_custom_command(OUTPUT source.c
   COMMAND "${PYTHON_EXECUTABLE}" "${script_path}"
"$<TARGET_PROPERTY:fake_target,INTERFACE_INCLUDE_DIRECTORIES>" >
source.c)

# Add a custom target to depend on source.c, as suggested
add_custom_target(depending_target)
set_target_properties(depending_target PROPERTIES
   IS_INTERROGATE 1)

# Compile the output
add_executable(output source.c)
# Make sure we depend on the custom target
add_dependencies(output depending_target)

I would change the above to the following:

add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/source.c
   COMMAND "${PYTHON_EXECUTABLE}" "${script_path}"
"$<TARGET_PROPERTY:fake_target,INTERFACE_INCLUDE_DIRECTORIES>" >
source.c)

# Add a custom target to depend on source.c, as suggested
add_custom_target(depending_target
DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/source.c)
set_target_properties(depending_target PROPERTIES
   IS_INTERROGATE 1)

# Compile the output
add_executable(output ${CMAKE_CURRENT_BINARY_DIR}/source.c)

Here are my reasons for the suggested changes.

* Absolute path when referring to source.c.  I am not sure whether
   this is stylistic or required, but this style works for me.

* Use DEPENDS to make the custom target depend on the custom command.
   This change is essential.

* Drop add_dependency.  The typical way you let CMake know that source
code is generated is with the GENERATED property which should take
care of all dependencies.  However,
<https://cmake.org/cmake/help/v3.11/prop_sf/GENERATED.html> implies to
me you do not even need to specify the GENERATED property for
${CMAKE_CURRENT_BINARY_DIR}/source.c.  So try the above and see, and
if it does not work try using the GENERATED property.

* Set a much higher minimum CMake version.  You mentioned CMake
2.8.12+ in your original post, but there are some fundamental
differences between CMake-2 and CMake-3, and in my opinion you are
just making a rod for your back if you try to make your build system
work for both.  For example, I have no idea whether the above approach
will work for 2.8.12.  Another constraint is early versions of CMake-3 were
frankly buggy.  So I use a minimum version of CMake-3.6.2 for all my
build systems, and that works well for me.  Note all modern
Linux distributions, Cygwin, MinGW-w64/MSYS2, HomeBrew, MacPorts, and
Fink all provide that version of CMake or higher, and Kitware provides
CMake for that version and higher for MSVC as well so that choice of minimum
CMake version should inconvenience very few of your potential users.

I hope these suggested changes help you toward your broader goal.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generator expressions: Identifying when they're used in a custom command?

Sam Edwards
Alan,

I'm kicking myself for leaving off the DEPENDS in add_custom_target as that is the most essential part of what you suggested. Bah!

I tried copying in your changes verbatim and I'm still left with an output that produces IS_INTERROGATE=0. This is on both 3.9.6 (my development machine) and 2.8.12 (my testing VM). Does your version of CMake produce IS_INTERROGATE=1 with your changes?

The rationale behind 2.8.12 is this is the version that ships with Ubuntu Trusty and will probably be what's present if a user is simply told to "install CMake" - although 3.5.1 is also available on that platform under the cmake3 package, so I might be able to justify a minimum version of either that or 3.0.1 (which is what's on backports-less Debian Jessie). I'd have to bring it up with the project maintainer to see, but in any case we're trying to follow a "stick with the same minimum version until we encounter a bug we can't work around, then bump the minimum to the version that fixes that bug" approach.

Thanks for your suggestions,
Sam

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generator expressions: Identifying when they're used in a custom command?

Elvis Stansvik
2018-02-19 21:03 GMT+01:00 Sam Edwards <[hidden email]>:

> Alan,
>
> I'm kicking myself for leaving off the DEPENDS in add_custom_target as that
> is the most essential part of what you suggested. Bah!
>
> I tried copying in your changes verbatim and I'm still left with an output
> that produces IS_INTERROGATE=0. This is on both 3.9.6 (my development
> machine) and 2.8.12 (my testing VM). Does your version of CMake produce
> IS_INTERROGATE=1 with your changes?
>
> The rationale behind 2.8.12 is this is the version that ships with Ubuntu
> Trusty and will probably be what's present if a user is simply told to
> "install CMake" - although 3.5.1 is also available on that platform under
> the cmake3 package, so I might be able to justify a minimum version of
> either that or 3.0.1 (which is what's on backports-less Debian Jessie). I'd
> have to bring it up with the project maintainer to see, but in any case
> we're trying to follow a "stick with the same minimum version until we
> encounter a bug we can't work around, then bump the minimum to the version
> that fixes that bug" approach.
>
> Thanks for your suggestions,

Hm, I'm not sure I'm enough of a CMake ninja to understand this, but,
when I read in the docs:

$<TARGET_PROPERTY:prop>
    Value of the property prop on the target on which the generator
expression is evaluated.

And then see:

set_target_properties(base_target PROPERTIES
  IS_INTERROGATE 0
  INTERFACE_INCLUDE_DIRECTORIES
"IS_INTERROGATE=$<BOOL:$<TARGET_PROPERTY:IS_INTERROGATE>>")

Correct me if I'm wrong, but I think that
$<TARGET_PROPERTY:IS_INTERROGATE> here will always be the value of
IS_INTERROGATE on the base_target (so 0), since that's the target on
which the generator expression is evaluated? Or?

I think that's where the 0 in your output comes from in the end?

Elvis

> Sam
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> https://cmake.org/mailman/listinfo/cmake
>
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Generator expressions: Identifying when they're used in a custom command?

Alan W. Irwin
In reply to this post by Sam Edwards
On 2018-02-19 13:03-0700 Sam Edwards wrote:

> Alan,
>
> I'm kicking myself for leaving off the DEPENDS in add_custom_target as that
> is the most essential part of what you suggested. Bah!
>
> I tried copying in your changes verbatim and I'm still left with an output
> that produces IS_INTERROGATE=0. This is on both 3.9.6 (my development
> machine) and 2.8.12 (my testing VM). Does your version of CMake produce
> IS_INTERROGATE=1 with your changes?

I don't want to get involved in too much further testing since the
overview of what you are attempting to do with generator expressions
is beyond my expertise.  However, I did build the following simple
project (with CMake-3.6.2)

cmake_minimum_required(VERSION 3.6.2 FATAL_ERROR)
project(test NONE)
add_custom_target(depending_target)
set_target_properties(depending_target PROPERTIES
   IS_INTERROGATE 1)
get_target_property(target_interrogate depending_target IS_INTERROGATE)
message(STATUS "target_interrogate = ${target_interrogate}")

The results were

-- target_interrogate = 1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/software/plplot/HEAD/build_dir/test_build

That result confirms that if you set that property on a custom target,
then that value is accessible via get_target_property (as expected).
So I don't know why that value is not currently accessible to you with
generator expressions, but for what it is worth (since this is beyond
my CMake expertise) I think it should be.  Anyhow, to debug this
further I suggest you simplify your test case even further (similar to
above) and compare generator expression results with
get_target_property results to try and understand what is going
on with the generator expression version.

>
> The rationale behind 2.8.12 is this is the version that ships with Ubuntu
> Trusty and will probably be what's present if a user is simply told to
> "install CMake" - although 3.5.1 is also available on that platform under
> the cmake3 package, so I might be able to justify a minimum version of
> either that or 3.0.1 (which is what's on backports-less Debian Jessie). I'd
> have to bring it up with the project maintainer to see, but in any case
> we're trying to follow a "stick with the same minimum version until we
> encounter a bug we can't work around, then bump the minimum to the version
> that fixes that bug" approach.

I would take the opposite approach, i.e., develop for the latest CMake, and
then once that completely works, push the minimum version to smaller
values to see how far you can get before you run into any difficulties.

In any case, I would definitely avoid early CMake 3 versions such as
3.0.1 for the reasons I mentioned.  I hauled that around on my back
for a while by working around its bugs with the PLplot build system,
but it was quite a lot of effort, and that build system became
somewhat simpler once I bumped the minimum cmake version to 3.6.2.

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake