CMake 3.9 change to dependencies of object compilation

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

CMake 3.9 change to dependencies of object compilation

Kevin Puetz
I saw the following in the CMake 3.9 release notes, but didn't immediately realize all the implications. Sorry for not catching this during the -rc phase...

. The Ninja generator has loosened the dependencies of object compilation. Object compilation now depends only on custom targets and custom commands associated with libraries on which the object's target depends and no longer depends on the libraries themselves. Source files in dependent targets may now compile without waiting for their targets' dependencies to link.

We have a few cases where the object compilation really does depend on the TARGET_FILE itself, e.g.
1. An RC compiler embedding the resulting file of a custom target (I think this one may still work, since custom targets appear to have been exempted from the change)

2. MSVC's #import construct which needs the indirect dependencies (dependencies of the #import-ed dependency) be registered, which is handled as part of the target using add_custom_command(TARGET foo POST_BUILD COMMAND ...)

So. I appreciate this loosening in most cases, it's greatly fixed some unnecessary stalls in our parallel build, and for dependencies acquired target_link_libraries this seems 100% correct to apply them only to the linker rule. But it seems the changes impacted add_dependencies as well, so now how do I now express a real dependency between a compile rule and another target (i.e. get a target listed at the same level as cmake_object_order_depend_*?

add_custom_command(OUTPUT... DEPENDS...) seemingly still allows target-level dependencies for a file compilation step, but I can't figure out what property has the same effect on a normal source file (that's going to be built by a "built-in" rule like CMAKE_RC_COMPILE_OBJECT, CMAKE_CXX_COMPILE_OBJECT, etc). AddFileDependencies(.) (aka OBJECT_DEPENDS) doesn't allow say it allows target-level dependencies, and it doesn't seem to work in practice either.

The best I've been able to come up with far requires an indirect SYMBOLIC output file to carry the dependency between a phony add_custom_command and the real source's OBJECT_DEPENDS

        ADD_CUSTOM_COMMAND(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/foo.depends DEPENDS bar)
        SET_PROPERTY(SOURCE ${CMAKE_CURRENT_BINARY_DIR}/foo.depends PROPERTY SYMBOLIC 1)
        SET_PROPERTY(SOURCE source.cpp APPEND PROPERTY OBJECT_DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/foo.depends)

That works, but it definitely feels more like a workaround than the intended solution. If the generators can handle this for add_custom_command, it seems like they should be able to handle it for language compile rules too. Any better suggestions?


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Ben Boeckel-2
> . The Ninja generator has loosened the dependencies of object
> compilation. Object compilation now depends only on custom targets and
> custom commands associated with libraries on which the object's target
> depends and no longer depends on the libraries themselves. Source
> files in dependent targets may now compile without waiting for their
> targets' dependencies to link.

Correct.

> We have a few cases where the object compilation really does depend on
> the TARGET_FILE itself, e.g.
> 1. An RC compiler embedding the resulting file of a custom target (I
> think this one may still work, since custom targets appear to have
> been exempted from the change)

Correct, though this issue:

    https://gitlab.kitware.com/cmake/cmake/issues/17097

requests that that be fixed as well (though that is backwards
compatible since the solution will likely involve ).

> 2. MSVC's #import construct which needs the indirect dependencies
> (dependencies of the #import-ed dependency) be registered, which is
> handled as part of the target using add_custom_command(TARGET foo
> POST_BUILD COMMAND ...)

So there's an issue here that there's a dependency between your build
rules which CMake doesn't know about (though I don't know #import well
enough, the docs don't state where the information *goes*). When adding
this custom command, you may use the `BYPRODUCTS` argument (introduced
in 3.2.0) to let CMake know what's going on here. It only affects Ninja,
but the other generators do target-level dependencies anyways. That
output can then be depended on via `OBJECT_DEPENDS` and the dependency
should link up properly.

If it instead gets registered somewhere in the aether (as far as CMake
is concerned), adding support for generator expressions to
`OBJECT_DEPENDS` so that `$<TARGET_FILE:tgt>` may be used there would be
the next solution.

Making `POST_BUILD` write out a stamp file would also work and then
using `OBJECT_DEPENDS` on that would also work.

--Ben
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Kevin Puetz
Thanks for the reply, questions/clarifications below.

> -----Original Message-----
> From: Ben Boeckel [mailto:[hidden email]]
> Sent: Thursday, August 03, 2017 10:59 AM
> To: [hidden email]
> Cc: Robert Maynard <[hidden email]>; Puetz Kevin A
> <[hidden email]>
> Subject: Re: CMake 3.9 change to dependencies of object compilation
>
> > . The Ninja generator has loosened the dependencies of object
> > compilation. Object compilation now depends only on custom targets and
> > custom commands associated with libraries on which the object's target
> > depends and no longer depends on the libraries themselves. Source
> > files in dependent targets may now compile without waiting for their
> > targets' dependencies to link.
>
> Correct.
>
> > We have a few cases where the object compilation really does depend on
> > the TARGET_FILE itself, e.g.
> > 1. An RC compiler embedding the resulting file of a custom target (I
> > think this one may still work, since custom targets appear to have
> > been exempted from the change)
>
> Correct, though this issue:
>
>     https://gitlab.kitware.com/cmake/cmake/issues/17097
>
> requests that that be fixed as well (though that is backwards compatible
> since the solution will likely involve ).

Sentence cut off? I assume you meant "will likely involve a new keyword"?

> > 2. MSVC's #import construct which needs the indirect dependencies
> > (dependencies of the #import-ed dependency) be registered, which is
> > handled as part of the target using add_custom_command(TARGET foo
> > POST_BUILD COMMAND ...)
>
> So there's an issue here that there's a dependency between your build rules
> which CMake doesn't know about (though I don't know #import well
> enough, the docs don't state where the information *goes*).

#import will load a COM typelib during preprocessing, possibly following registry keys to locate other typelibs which the specified one refers to. It will have the byproduct of creating .tlh/.tli files next to the other compiler outputs (e.g. .o file) Arguably the .tlh/.tli files should be listed in OBJECT_OUTPUTS, but I can't because I don't know their location; CMake doesn't have a variable/property/generator expression that reveals where it's going to place the object files (i.e. /Fo$out), so I don't know where they will end up. Luckily the .tlh/.tli files aren't important to list for dependency resolution anyway, because the #import also automatically #includes the just-generated headers, (though this is not mentioned in /showIncludes). So CMake is at least *consistently* unaware of these files, and they get regenerated any time they would have been read so it doesn't really need to know.

The important missing dependency is the one between creating/regstering the typelib (we'll call this target COMServer) and the #import that will read it in a source file in another target (we'll call it COMClient).  I have a call add_custom_command(TARGET COMServer POST_BUILD COMMAND regsvr32 $<TARGET_FILE: COMServer>), which will create the registry keys under HKEY_CLASSES_ROOT. This needs to happen before the source file in COMClient can preprocess the #import successfully. Prior to CMake 3.9, I could inform CMake of this by just using add_dependencies(COMClient COMServer) to tell CMake that it couldn't build (any of) Client until Server had been built (and thus its POST_BUILD had also run to register it). But in 3.9, add_dependencies has changed in meaning; although the documentation still says "to ensure that they build before <target> does", in practice this now only means "to ensure that they build before <target> *links*"; these edges do not apply to object compilation or add_
 custom_command rules.

add_custom_command is no problem; it already had a DEPENDS argument that allows target-level dependencies, and arguably such dependencies needed to be stated there anyway since an add_custom_command output can get reused by multiple targets in the same subdir. But object compilation is a problem because there's nowhere to add them per-source, and add_dependencies doesn't work anymore to add them per-target.

> When adding
> this custom command, you may use the `BYPRODUCTS` argument
> (introduced in 3.2.0) to let CMake know what's going on here. It only affects
> Ninja, but the other generators do target-level dependencies anyways. That
> output can then be depended on via `OBJECT_DEPENDS` and the
> dependency should link up properly.

There is not an explicit file output, though I could do the usual workaround of a stamp/witness file listed in BYPRODUCTS to the add_custom_command(TARGET ... POST_BUILD ...). But I don't think that will work with most generators, since CMake doesn't generally allow file-level depends to set the order in which targets are built. I suppose it might work out in practice for ninja since that writes a monolithic set of rules, but conditional code where I have to peek at CMAKE_GENERATOR and use BYPRODUCTS/OBJECT_DEPENDS for ninja and add_dependencies for other generators seems like the sort of thing this list would tell me not to do :-)

And even for ninja I think I'd have to be making undocumented assumptions about the binary dir layout to refer to my witness file that was generated in a different subdir's CMakelists.txt.

> If it instead gets registered somewhere in the aether (as far as CMake is
> concerned), adding support for generator expressions to `OBJECT_DEPENDS`
> so that `$<TARGET_FILE:tgt>` may be used there would be the next solution.

Yes, the dependency in question for #import is on information "somewhere in the aether" (or rather the Win32 registry).

Supporting $<TARGET_FILE> does in OBJECT_DEPENDS would be a great solution for my first use case of a embedding that file in a resource. But I don't think that helps with #import, since I don't actually want to read the $<TARGET_FILE>, I just want the post-build that registers it to have run. Also, in the cases of .tlb files that are *not* embedded in DLL resources, the target in question is going to be an add_custom_target from another subdirectory; the .tlb file is built by an add_custom_command(OUTPUT...) but this rule gets emitted in an add_custom_target that depends on this file to build it and then registers it. If each subdir had the add_custom_command instead of using an intermediate target, multiple targets would each end up with their own copy of the rule to build the .tlb file, leading to race conditions where they all try to build it at once and get file-in-use errors (they can't just build individual copies, because it has to end up with a unique key referencing the .tlb
  path in the win32 registry).

You're not currently allowed to use $<TARGET_FILE:x> on UTILITY targets even if the LOCATION property has been set (it's blocked in TargetFilesystemArtifact::Evaluate with "Target <x> is not an executable or library"). Maybe that could be changed as well (which would be nice), but it seems like if one is adding support for $<TARGET_FILE> generator expressions in OBJECT_DEPENDS (which implies supporting the generator context and context->DependTargets), it seems like you as well go the rest of the way and just treat them completely the same as the DEPENDS argument to add_custom_command, allowing both file and target dependencies to be listed in the first place.

> Making `POST_BUILD` write out a stamp file would also work and then using
> `OBJECT_DEPENDS` on that would also work.

No, as above I don't think that would be legal across subdirs, at least in the context of a CMake file that's supposed to work with various generators. Feel free to correct me if I'm wrong about that...

> --Ben
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Ben Boeckel-2
On Fri, Aug 04, 2017 at 17:35:53 +0000, Puetz Kevin A wrote:
> Thanks for the reply, questions/clarifications below.
> > requests that that be fixed as well (though that is backwards compatible
> > since the solution will likely involve ).
>
> Sentence cut off? I assume you meant "will likely involve a new keyword"?

Yep, sorry.

> > > 2. MSVC's #import construct which needs the indirect dependencies
> > > (dependencies of the #import-ed dependency) be registered, which is
> > > handled as part of the target using add_custom_command(TARGET foo
> > > POST_BUILD COMMAND ...)
> >
> > So there's an issue here that there's a dependency between your build rules
> > which CMake doesn't know about (though I don't know #import well
> > enough, the docs don't state where the information *goes*).
>
> #import will load a COM typelib during preprocessing, possibly
> following registry keys to locate other typelibs which the specified
> one refers to. It will have the byproduct of creating .tlh/.tli files
> next to the other compiler outputs (e.g. .o file) Arguably the
> .tlh/.tli files should be listed in OBJECT_OUTPUTS, but I can't
> because I don't know their location; CMake doesn't have a
> variable/property/generator expression that reveals where it's going
> to place the object files (i.e. /Fo$out), so I don't know where they
> will end up. Luckily the .tlh/.tli files aren't important to list for
> dependency resolution anyway, because the #import also automatically
> #includes the just-generated headers, (though this is not mentioned in
> /showIncludes). So CMake is at least *consistently* unaware of these
> files, and they get regenerated any time they would have been read so
> it doesn't really need to know.

OK, a genex for where object outputs may be useful anyways. I think
there's something along those lines with Cuda's PTX file generation?

> The important missing dependency is the one between
> creating/regstering the typelib (we'll call this target COMServer) and
> the #import that will read it in a source file in another target
> (we'll call it COMClient).  I have a call add_custom_command(TARGET
> COMServer POST_BUILD COMMAND regsvr32 $<TARGET_FILE: COMServer>),
> which will create the registry keys under HKEY_CLASSES_ROOT. This
> needs to happen before the source file in COMClient can preprocess the
> #import successfully. Prior to CMake 3.9, I could inform CMake of this
> by just using add_dependencies(COMClient COMServer) to tell CMake that
> it couldn't build (any of) Client until Server had been built (and
> thus its POST_BUILD had also run to register it). But in 3.9,
> add_dependencies has changed in meaning; although the documentation
> still says "to ensure that they build before <target> does", in
> practice this now only means "to ensure that they build before
> <target> *links*"; these edges do not apply to object compilation or
> add_custom_command rules.
>
> add_custom_command is no problem; it already had a DEPENDS argument
> that allows target-level dependencies, and arguably such dependencies
> needed to be stated there anyway since an add_custom_command output
> can get reused by multiple targets in the same subdir. But object
> compilation is a problem because there's nowhere to add them
> per-source, and add_dependencies doesn't work anymore to add them
> per-target.

It sounds like the logic may need fixing then. Do you have an example
case where add_dependencies doesn't work anymore in Ninja?

> > When adding
> > this custom command, you may use the `BYPRODUCTS` argument
> > (introduced in 3.2.0) to let CMake know what's going on here. It only affects
> > Ninja, but the other generators do target-level dependencies anyways. That
> > output can then be depended on via `OBJECT_DEPENDS` and the
> > dependency should link up properly.
>
> There is not an explicit file output, though I could do the usual
> workaround of a stamp/witness file listed in BYPRODUCTS to the
> add_custom_command(TARGET ... POST_BUILD ...). But I don't think that
> will work with most generators, since CMake doesn't generally allow
> file-level depends to set the order in which targets are built. I
> suppose it might work out in practice for ninja since that writes a
> monolithic set of rules, but conditional code where I have to peek at
> CMAKE_GENERATOR and use BYPRODUCTS/OBJECT_DEPENDS for ninja and
> add_dependencies for other generators seems like the sort of thing
> this list would tell me not to do :-)

Well, other generators are generally target-ordered anyways. Ninja is
the oddball here (which is why it's the only one to get the feature). I
don't know the effect it'd have in other generators, but I feel like I'd
be surprised if it *broke* them since excess dependencies (usually) only
result in either slower builds or circular dependency loops and Ninja
complains loudly about the latter. And since BYPRODUCTS only affects
Ninja, if BYPRODUCTS is used, other generators shouldn't care anyways.

> And even for ninja I think I'd have to be making undocumented
> assumptions about the binary dir layout to refer to my witness file
> that was generated in a different subdir's CMakelists.txt.

There's nothing stopping the witness files couldn't all be under a
single directory (such as ${CMAKE_BINARY_DIR}/CMakeFiles/tlb).

> > If it instead gets registered somewhere in the aether (as far as CMake is
> > concerned), adding support for generator expressions to `OBJECT_DEPENDS`
> > so that `$<TARGET_FILE:tgt>` may be used there would be the next solution.
>
> Yes, the dependency in question for #import is on information
> "somewhere in the aether" (or rather the Win32 registry).
>
> Supporting $<TARGET_FILE> does in OBJECT_DEPENDS would be a great
> solution for my first use case of a embedding that file in a resource.
> But I don't think that helps with #import, since I don't actually want
> to read the $<TARGET_FILE>, I just want the post-build that registers
> it to have run.

POST_BUILD rules are attached to the target, so depending on the target
also guarantees that the POST_BUILD command(s) have run as well.

> Also, in the cases of .tlb files that are *not* embedded in DLL
> resources, the target in question is going to be an add_custom_target
> from another subdirectory; the .tlb file is built by an
> add_custom_command(OUTPUT...) but this rule gets emitted in an
> add_custom_target that depends on this file to build it and then
> registers it. If each subdir had the add_custom_command instead of
> using an intermediate target, multiple targets would each end up with
> their own copy of the rule to build the .tlb file, leading to race
> conditions where they all try to build it at once and get file-in-use
> errors (they can't just build individual copies, because it has to end
> up with a unique key referencing the .tlb path in the win32 regist>
> ry).

Yeah, there should be just one .tlb rule writer. Usually I handle that
by collecting information in global properties and writing a rule at the
end to handle all of them.

> You're not currently allowed to use $<TARGET_FILE:x> on UTILITY
> targets even if the LOCATION property has been set (it's blocked in
> TargetFilesystemArtifact::Evaluate with "Target <x> is not an
> executable or library"). Maybe that could be changed as well (which
> would be nice), but it seems like if one is adding support for
> $<TARGET_FILE> generator expressions in OBJECT_DEPENDS (which implies
> supporting the generator context and context->DependTargets), it seems
> like you as well go the rest of the way and just treat them completely
> the same as the DEPENDS argument to add_custom_command, allowing both
> file and target dependencies to be listed in the first place.

That sounds like a likely path to follow when supporting genexes in
OBJECT_DEPENDS.

> > Making `POST_BUILD` write out a stamp file would also work and then using
> > `OBJECT_DEPENDS` on that would also work.
>
> No, as above I don't think that would be legal across subdirs, at
> least in the context of a CMake file that's supposed to work with
> various generators. Feel free to correct me if I'm wrong about that...

Experiments would be more useful. add_custom_* have some of the most
complicated interaction semantics in CMake. I can't keep all of them
straight all the time (usually I rediscover them when necessary; I
should probably write up some docs next time I need to do so).

--Ben
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Robert Maynard
You can find the location for object files by using
$<TARGET_OBJECTS:${name}>. This can be used as the DEPENDS of a custom
command. I would not try using OBJECT_OUTPUTS as IIRC that is only
used by the makefile generator.

On Fri, Aug 4, 2017 at 1:55 PM, Ben Boeckel <[hidden email]> wrote:

> On Fri, Aug 04, 2017 at 17:35:53 +0000, Puetz Kevin A wrote:
>> Thanks for the reply, questions/clarifications below.
>> > requests that that be fixed as well (though that is backwards compatible
>> > since the solution will likely involve ).
>>
>> Sentence cut off? I assume you meant "will likely involve a new keyword"?
>
> Yep, sorry.
>
>> > > 2. MSVC's #import construct which needs the indirect dependencies
>> > > (dependencies of the #import-ed dependency) be registered, which is
>> > > handled as part of the target using add_custom_command(TARGET foo
>> > > POST_BUILD COMMAND ...)
>> >
>> > So there's an issue here that there's a dependency between your build rules
>> > which CMake doesn't know about (though I don't know #import well
>> > enough, the docs don't state where the information *goes*).
>>
>> #import will load a COM typelib during preprocessing, possibly
>> following registry keys to locate other typelibs which the specified
>> one refers to. It will have the byproduct of creating .tlh/.tli files
>> next to the other compiler outputs (e.g. .o file) Arguably the
>> .tlh/.tli files should be listed in OBJECT_OUTPUTS, but I can't
>> because I don't know their location; CMake doesn't have a
>> variable/property/generator expression that reveals where it's going
>> to place the object files (i.e. /Fo$out), so I don't know where they
>> will end up. Luckily the .tlh/.tli files aren't important to list for
>> dependency resolution anyway, because the #import also automatically
>> #includes the just-generated headers, (though this is not mentioned in
>> /showIncludes). So CMake is at least *consistently* unaware of these
>> files, and they get regenerated any time they would have been read so
>> it doesn't really need to know.
>
> OK, a genex for where object outputs may be useful anyways. I think
> there's something along those lines with Cuda's PTX file generation?
>
>> The important missing dependency is the one between
>> creating/regstering the typelib (we'll call this target COMServer) and
>> the #import that will read it in a source file in another target
>> (we'll call it COMClient).  I have a call add_custom_command(TARGET
>> COMServer POST_BUILD COMMAND regsvr32 $<TARGET_FILE: COMServer>),
>> which will create the registry keys under HKEY_CLASSES_ROOT. This
>> needs to happen before the source file in COMClient can preprocess the
>> #import successfully. Prior to CMake 3.9, I could inform CMake of this
>> by just using add_dependencies(COMClient COMServer) to tell CMake that
>> it couldn't build (any of) Client until Server had been built (and
>> thus its POST_BUILD had also run to register it). But in 3.9,
>> add_dependencies has changed in meaning; although the documentation
>> still says "to ensure that they build before <target> does", in
>> practice this now only means "to ensure that they build before
>> <target> *links*"; these edges do not apply to object compilation or
>> add_custom_command rules.
>>
>> add_custom_command is no problem; it already had a DEPENDS argument
>> that allows target-level dependencies, and arguably such dependencies
>> needed to be stated there anyway since an add_custom_command output
>> can get reused by multiple targets in the same subdir. But object
>> compilation is a problem because there's nowhere to add them
>> per-source, and add_dependencies doesn't work anymore to add them
>> per-target.
>
> It sounds like the logic may need fixing then. Do you have an example
> case where add_dependencies doesn't work anymore in Ninja?
>
>> > When adding
>> > this custom command, you may use the `BYPRODUCTS` argument
>> > (introduced in 3.2.0) to let CMake know what's going on here. It only affects
>> > Ninja, but the other generators do target-level dependencies anyways. That
>> > output can then be depended on via `OBJECT_DEPENDS` and the
>> > dependency should link up properly.
>>
>> There is not an explicit file output, though I could do the usual
>> workaround of a stamp/witness file listed in BYPRODUCTS to the
>> add_custom_command(TARGET ... POST_BUILD ...). But I don't think that
>> will work with most generators, since CMake doesn't generally allow
>> file-level depends to set the order in which targets are built. I
>> suppose it might work out in practice for ninja since that writes a
>> monolithic set of rules, but conditional code where I have to peek at
>> CMAKE_GENERATOR and use BYPRODUCTS/OBJECT_DEPENDS for ninja and
>> add_dependencies for other generators seems like the sort of thing
>> this list would tell me not to do :-)
>
> Well, other generators are generally target-ordered anyways. Ninja is
> the oddball here (which is why it's the only one to get the feature). I
> don't know the effect it'd have in other generators, but I feel like I'd
> be surprised if it *broke* them since excess dependencies (usually) only
> result in either slower builds or circular dependency loops and Ninja
> complains loudly about the latter. And since BYPRODUCTS only affects
> Ninja, if BYPRODUCTS is used, other generators shouldn't care anyways.
>
>> And even for ninja I think I'd have to be making undocumented
>> assumptions about the binary dir layout to refer to my witness file
>> that was generated in a different subdir's CMakelists.txt.
>
> There's nothing stopping the witness files couldn't all be under a
> single directory (such as ${CMAKE_BINARY_DIR}/CMakeFiles/tlb).
>
>> > If it instead gets registered somewhere in the aether (as far as CMake is
>> > concerned), adding support for generator expressions to `OBJECT_DEPENDS`
>> > so that `$<TARGET_FILE:tgt>` may be used there would be the next solution.
>>
>> Yes, the dependency in question for #import is on information
>> "somewhere in the aether" (or rather the Win32 registry).
>>
>> Supporting $<TARGET_FILE> does in OBJECT_DEPENDS would be a great
>> solution for my first use case of a embedding that file in a resource.
>> But I don't think that helps with #import, since I don't actually want
>> to read the $<TARGET_FILE>, I just want the post-build that registers
>> it to have run.
>
> POST_BUILD rules are attached to the target, so depending on the target
> also guarantees that the POST_BUILD command(s) have run as well.
>
>> Also, in the cases of .tlb files that are *not* embedded in DLL
>> resources, the target in question is going to be an add_custom_target
>> from another subdirectory; the .tlb file is built by an
>> add_custom_command(OUTPUT...) but this rule gets emitted in an
>> add_custom_target that depends on this file to build it and then
>> registers it. If each subdir had the add_custom_command instead of
>> using an intermediate target, multiple targets would each end up with
>> their own copy of the rule to build the .tlb file, leading to race
>> conditions where they all try to build it at once and get file-in-use
>> errors (they can't just build individual copies, because it has to end
>> up with a unique key referencing the .tlb path in the win32 regist>
>> ry).
>
> Yeah, there should be just one .tlb rule writer. Usually I handle that
> by collecting information in global properties and writing a rule at the
> end to handle all of them.
>
>> You're not currently allowed to use $<TARGET_FILE:x> on UTILITY
>> targets even if the LOCATION property has been set (it's blocked in
>> TargetFilesystemArtifact::Evaluate with "Target <x> is not an
>> executable or library"). Maybe that could be changed as well (which
>> would be nice), but it seems like if one is adding support for
>> $<TARGET_FILE> generator expressions in OBJECT_DEPENDS (which implies
>> supporting the generator context and context->DependTargets), it seems
>> like you as well go the rest of the way and just treat them completely
>> the same as the DEPENDS argument to add_custom_command, allowing both
>> file and target dependencies to be listed in the first place.
>
> That sounds like a likely path to follow when supporting genexes in
> OBJECT_DEPENDS.
>
>> > Making `POST_BUILD` write out a stamp file would also work and then using
>> > `OBJECT_DEPENDS` on that would also work.
>>
>> No, as above I don't think that would be legal across subdirs, at
>> least in the context of a CMake file that's supposed to work with
>> various generators. Feel free to correct me if I'm wrong about that...
>
> Experiments would be more useful. add_custom_* have some of the most
> complicated interaction semantics in CMake. I can't keep all of them
> straight all the time (usually I rediscover them when necessary; I
> should probably write up some docs next time I need to do so).
>
> --Ben
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Kevin Puetz
> -----Original Message-----
> From: Robert Maynard [mailto:[hidden email]]
> Sent: Friday, August 04, 2017 1:03 PM
> To: Ben Boeckel <[hidden email]>
> Cc: Puetz Kevin A <[hidden email]>; [hidden email]
> Subject: Re: CMake 3.9 change to dependencies of object compilation
>
> You can find the location for object files by using
> $<TARGET_OBJECTS:${name}>. This can be used as the DEPENDS of a custom
> command. I would not try using OBJECT_OUTPUTS as IIRC that is only used by
> the makefile generator.

Isn't TARGET_OBJECTS  only for OBJECT libraries? I don't think it would give me a way to get the paths for the .o file that's going to result from a particular source file (in order to locate other files the compiler is going to generate alongside a .o file).

But and in any case, this was a little bit of an aside, just because Ben asked what #import generated. It would feel *correct* to inform that the .tlh/.tli files were OBJECT_OUTPUTS, but nothing in the processing really *needs* to know. The problem I'm having with 3.9 is the input dependency on registry keys (previously modeled as a dependency on the target whose POST_BUILD would create them), not on the outputs.
 
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Kevin Puetz
In reply to this post by Ben Boeckel-2
> -----Original Message-----
> From: Ben Boeckel [mailto:[hidden email]]
> Sent: Friday, August 04, 2017 12:55 PM
> To: Puetz Kevin A <[hidden email]>
> Cc: [hidden email]; Robert Maynard <[hidden email]>
> Subject: Re: CMake 3.9 change to dependencies of object compilation
>
> On Fri, Aug 04, 2017 at 17:35:53 +0000, Puetz Kevin A wrote:

> > > > 2. MSVC's #import construct which needs the indirect dependencies
> > > > (dependencies of the #import-ed dependency) be registered, which
> > > > is handled as part of the target using add_custom_command(TARGET
> > > > foo POST_BUILD COMMAND ...)
> > >
> > > So there's an issue here that there's a dependency between your
> > > build rules which CMake doesn't know about (though I don't know
> > > #import well enough, the docs don't state where the information
> *goes*).
> >
> > #import will load a COM typelib during preprocessing, possibly
> > following registry keys to locate other typelibs which the specified
> > one refers to. It will have the byproduct of creating .tlh/.tli files
> > next to the other compiler outputs (e.g. .o file) Arguably the
> > .tlh/.tli files should be listed in OBJECT_OUTPUTS, but I can't
> > because I don't know their location; CMake doesn't have a
> > variable/property/generator expression that reveals where it's going
> > to place the object files (i.e. /Fo$out), so I don't know where they
> > will end up. Luckily the .tlh/.tli files aren't important to list for
> > dependency resolution anyway, because the #import also automatically
> > #includes the just-generated headers, (though this is not mentioned in
> > /showIncludes). So CMake is at least *consistently* unaware of these
> > files, and they get regenerated any time they would have been read so
> > it doesn't really need to know.
>
> OK, a genex for where object outputs may be useful anyways. I think there's
> something along those lines with Cuda's PTX file generation?

It would also be really nice for things like precompiled headers; I have some custom commands where it really feels right to put their outputs with the other object files for a target (this automatically getting things right for multi-configuration generators and such), but can't because there's no expression for that. Something like $<TARGET_OBJ_DIR:tgt> would be very welcome.

> > The important missing dependency is the one between
> > creating/regstering the typelib (we'll call this target COMServer) and
> > the #import that will read it in a source file in another target
> > (we'll call it COMClient).  I have a call add_custom_command(TARGET
> > COMServer POST_BUILD COMMAND regsvr32 $<TARGET_FILE:
> COMServer>),
> > which will create the registry keys under HKEY_CLASSES_ROOT. This
> > needs to happen before the source file in COMClient can preprocess the
> > #import successfully. Prior to CMake 3.9, I could inform CMake of this
> > by just using add_dependencies(COMClient COMServer) to tell CMake that
> > it couldn't build (any of) Client until Server had been built (and
> > thus its POST_BUILD had also run to register it). But in 3.9,
> > add_dependencies has changed in meaning; although the documentation
> > still says "to ensure that they build before <target> does", in
> > practice this now only means "to ensure that they build before
> > <target> *links*"; these edges do not apply to object compilation or
> > add_custom_command rules.
> >
> > add_custom_command is no problem; it already had a DEPENDS argument
> > that allows target-level dependencies, and arguably such dependencies
> > needed to be stated there anyway since an add_custom_command output
> > can get reused by multiple targets in the same subdir. But object
> > compilation is a problem because there's nowhere to add them
> > per-source, and add_dependencies doesn't work anymore to add them
> > per-target.
>
> It sounds like the logic may need fixing then. Do you have an example case
> where add_dependencies doesn't work anymore in Ninja?

CMakeLists.txt:
cmake_minimum_required(VERSION 3.7)

add_library(A SHARED a.c)

add_custom_command(TARGET A POST_BUILD
        COMMENT "hello A"
        COMMAND ${CMAKE_COMMAND} -E sleep 3
        COMMAND ${CMAKE_COMMAND} -E echo "hello A")

add_custom_command(OUTPUT b.custom
        COMMENT "hello B"
        COMMAND ${CMAKE_COMMAND} -E touch b.custom)

add_executable(B b.c b.custom)
add_dependencies(B A)

a.c:
void foo() {}

b.c:
int main() { return 0; }

In CMake 3.7:

build cmake_order_depends_target_B: phony || A.dll b.custom
build b.custom: CUSTOM_COMMAND || A.dll
build CMakeFiles\B.dir\b.c.obj: C_COMPILER__B C$:\Users\re41236\Desktop\test\cmake\b.c || cmake_order_depends_target_B
build B.exe: C_EXECUTABLE_LINKER__B CMakeFiles\B.dir\b.c.obj || A.dll

In CMake 3.9:
build cmake_object_order_depends_target_B: phony || b.custom cmake_object_order_depends_target_A
build CMakeFiles\B.dir\b.c.obj: C_COMPILER__B C$:\Users\re41236\Desktop\test\cmake\b.c || cmake_object_order_depends_target_B
build b.custom: CUSTOM_COMMAND || A.dll
build B.exe: C_EXECUTABLE_LINKER__B CMakeFiles\B.dir\b.c.obj || A.dll

So in 3.7, the add_dependencies(B A) put A.dll as an order-only dependency of all the rules that were part of B; nothing in B builds until everything in A does.
In 3.9 only the custom commands and link rules get it, the object rules don't. And there doesn't seem to be a way to explicitly get an order-only target dependency into the object rules in the (rare) cases where it actually was needed, like #import. So not only is it a breaking change, it’s not readily fixed.

Admittedly right now this is pretty much limited to weird compiler (mis-)features like MSVC's #import that reference input besides the source/headers. Although it might become more common if the C++ modules TS ever catches on (you'll need the .ifc file generated by compiling the dependency before compiling objects that use it)

> > > When adding
> > > this custom command, you may use the `BYPRODUCTS` argument
> > > (introduced in 3.2.0) to let CMake know what's going on here. It
> > > only affects Ninja, but the other generators do target-level
> > > dependencies anyways. That output can then be depended on via
> > > `OBJECT_DEPENDS` and the dependency should link up properly.
> >
> > There is not an explicit file output, though I could do the usual
> > workaround of a stamp/witness file listed in BYPRODUCTS to the
> > add_custom_command(TARGET ... POST_BUILD ...). But I don't think that
> > will work with most generators, since CMake doesn't generally allow
> > file-level depends to set the order in which targets are built. I
> > suppose it might work out in practice for ninja since that writes a
> > monolithic set of rules, but conditional code where I have to peek at
> > CMAKE_GENERATOR and use BYPRODUCTS/OBJECT_DEPENDS for ninja
> and
> > add_dependencies for other generators seems like the sort of thing
> > this list would tell me not to do :-)
>
> Well, other generators are generally target-ordered anyways. Ninja is the
> oddball here (which is why it's the only one to get the feature). I don't know
> the effect it'd have in other generators, but I feel like I'd be surprised if it
> *broke* them since excess dependencies (usually) only result in either
> slower builds or circular dependency loops and Ninja complains loudly about
> the latter. And since BYPRODUCTS only affects Ninja, if BYPRODUCTS is used,
> other generators shouldn't care anyways.

True, I suppose I don’t need to peek at CMAKE_GENERATOR, I could just do it both ways all the time. As long as nobody comes up with a generator that can separate compile vs link dependencies, but still can't handle cross-subdir file dependencies.

> > And even for ninja I think I'd have to be making undocumented
> > assumptions about the binary dir layout to refer to my witness file
> > that was generated in a different subdir's CMakelists.txt.
>
> There's nothing stopping the witness files couldn't all be under a single
> directory (such as ${CMAKE_BINARY_DIR}/CMakeFiles/tlb).

Ok, good point. They don't have to be alongside the target's other artifacts.

> > > If it instead gets registered somewhere in the aether (as far as
> > > CMake is concerned), adding support for generator expressions to
> > > `OBJECT_DEPENDS` so that `$<TARGET_FILE:tgt>` may be used there
> would be the next solution.
> >
> > Yes, the dependency in question for #import is on information
> > "somewhere in the aether" (or rather the Win32 registry).
> >
> > Supporting $<TARGET_FILE> does in OBJECT_DEPENDS would be a great
> > solution for my first use case of a embedding that file in a resource.
> > But I don't think that helps with #import, since I don't actually want
> > to read the $<TARGET_FILE>, I just want the post-build that registers
> > it to have run.
>
> POST_BUILD rules are attached to the target, so depending on the target also
> guarantees that the POST_BUILD command(s) have run as well.

Right, which is how I did it before. What I was saying (not very clearly) is that I only want the target to have been registered (so it's POST_BUILD has run), I don't care if it's newer. I only care if the .tlb file is newer, and I can do that with OBJECT_DEPENDS already. So the target part ought to be an order-only dependency (as it was in 3.8). Depending on $<TARGET_FILE> is sufficient, but actually too strong.

> > Also, in the cases of .tlb files that are *not* embedded in DLL
> > resources, the target in question is going to be an add_custom_target
> > from another subdirectory; the .tlb file is built by an
> > add_custom_command(OUTPUT...) but this rule gets emitted in an
> > add_custom_target that depends on this file to build it and then
> > registers it. If each subdir had the add_custom_command instead of
> > using an intermediate target, multiple targets would each end up with
> > their own copy of the rule to build the .tlb file, leading to race
> > conditions where they all try to build it at once and get file-in-use
> > errors (they can't just build individual copies, because it has to end
> > up with a unique key referencing the .tlb path in the win32 regist>
> > ry).
>
> Yeah, there should be just one .tlb rule writer. Usually I handle that by
> collecting information in global properties and writing a rule at the end to
> handle all of them.

Yeah. It would be really cool to have an end-of-input callback so include files that define their own commands could get a callback hook to write such "rule at the end" functions :-).

> > You're not currently allowed to use $<TARGET_FILE:x> on UTILITY
> > targets even if the LOCATION property has been set (it's blocked in
> > TargetFilesystemArtifact::Evaluate with "Target <x> is not an
> > executable or library"). Maybe that could be changed as well (which
> > would be nice), but it seems like if one is adding support for
> > $<TARGET_FILE> generator expressions in OBJECT_DEPENDS (which
> implies
> > supporting the generator context and context->DependTargets), it seems
> > like you as well go the rest of the way and just treat them completely
> > the same as the DEPENDS argument to add_custom_command, allowing
> both
> > file and target dependencies to be listed in the first place.
>
> That sounds like a likely path to follow when supporting genexes in
> OBJECT_DEPENDS.

This would definitely solve my complaint; I'd be able to put target depends on an object rule in the (rare) case that it really needs them.

> > > Making `POST_BUILD` write out a stamp file would also work and then
> > > using `OBJECT_DEPENDS` on that would also work.
> >
> > No, as above I don't think that would be legal across subdirs, at
> > least in the context of a CMake file that's supposed to work with
> > various generators. Feel free to correct me if I'm wrong about that...
>
> Experiments would be more useful. add_custom_* have some of the most
> complicated interaction semantics in CMake. I can't keep all of them straight
> all the time (usually I rediscover them when necessary; I should probably
> write up some docs next time I need to do so).
>
> --Ben
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CMake 3.9 change to dependencies of object compilation

Robert Maynard
I see no reason that we couldn't provide a TARGET_OBJ_DIR:tgt
generator expression that states where the objects reside. Opening
TARGET_OBJ_DIR and TARGET_OBJECTS to be evaluated for any target would
require some work, and an interested CMake developer.

FYI Due to XCode limitations we can't allow people to specify where
object files should be placed for a library :( .

On Fri, Aug 4, 2017 at 3:46 PM, Puetz Kevin A <[hidden email]> wrote:

>> -----Original Message-----
>> From: Ben Boeckel [mailto:[hidden email]]
>> Sent: Friday, August 04, 2017 12:55 PM
>> To: Puetz Kevin A <[hidden email]>
>> Cc: [hidden email]; Robert Maynard <[hidden email]>
>> Subject: Re: CMake 3.9 change to dependencies of object compilation
>>
>> On Fri, Aug 04, 2017 at 17:35:53 +0000, Puetz Kevin A wrote:
>
>> > > > 2. MSVC's #import construct which needs the indirect dependencies
>> > > > (dependencies of the #import-ed dependency) be registered, which
>> > > > is handled as part of the target using add_custom_command(TARGET
>> > > > foo POST_BUILD COMMAND ...)
>> > >
>> > > So there's an issue here that there's a dependency between your
>> > > build rules which CMake doesn't know about (though I don't know
>> > > #import well enough, the docs don't state where the information
>> *goes*).
>> >
>> > #import will load a COM typelib during preprocessing, possibly
>> > following registry keys to locate other typelibs which the specified
>> > one refers to. It will have the byproduct of creating .tlh/.tli files
>> > next to the other compiler outputs (e.g. .o file) Arguably the
>> > .tlh/.tli files should be listed in OBJECT_OUTPUTS, but I can't
>> > because I don't know their location; CMake doesn't have a
>> > variable/property/generator expression that reveals where it's going
>> > to place the object files (i.e. /Fo$out), so I don't know where they
>> > will end up. Luckily the .tlh/.tli files aren't important to list for
>> > dependency resolution anyway, because the #import also automatically
>> > #includes the just-generated headers, (though this is not mentioned in
>> > /showIncludes). So CMake is at least *consistently* unaware of these
>> > files, and they get regenerated any time they would have been read so
>> > it doesn't really need to know.
>>
>> OK, a genex for where object outputs may be useful anyways. I think there's
>> something along those lines with Cuda's PTX file generation?
>
> It would also be really nice for things like precompiled headers; I have some custom commands where it really feels right to put their outputs with the other object files for a target (this automatically getting things right for multi-configuration generators and such), but can't because there's no expression for that. Something like $<TARGET_OBJ_DIR:tgt> would be very welcome.
>
>> > The important missing dependency is the one between
>> > creating/regstering the typelib (we'll call this target COMServer) and
>> > the #import that will read it in a source file in another target
>> > (we'll call it COMClient).  I have a call add_custom_command(TARGET
>> > COMServer POST_BUILD COMMAND regsvr32 $<TARGET_FILE:
>> COMServer>),
>> > which will create the registry keys under HKEY_CLASSES_ROOT. This
>> > needs to happen before the source file in COMClient can preprocess the
>> > #import successfully. Prior to CMake 3.9, I could inform CMake of this
>> > by just using add_dependencies(COMClient COMServer) to tell CMake that
>> > it couldn't build (any of) Client until Server had been built (and
>> > thus its POST_BUILD had also run to register it). But in 3.9,
>> > add_dependencies has changed in meaning; although the documentation
>> > still says "to ensure that they build before <target> does", in
>> > practice this now only means "to ensure that they build before
>> > <target> *links*"; these edges do not apply to object compilation or
>> > add_custom_command rules.
>> >
>> > add_custom_command is no problem; it already had a DEPENDS argument
>> > that allows target-level dependencies, and arguably such dependencies
>> > needed to be stated there anyway since an add_custom_command output
>> > can get reused by multiple targets in the same subdir. But object
>> > compilation is a problem because there's nowhere to add them
>> > per-source, and add_dependencies doesn't work anymore to add them
>> > per-target.
>>
>> It sounds like the logic may need fixing then. Do you have an example case
>> where add_dependencies doesn't work anymore in Ninja?
>
> CMakeLists.txt:
> cmake_minimum_required(VERSION 3.7)
>
> add_library(A SHARED a.c)
>
> add_custom_command(TARGET A POST_BUILD
>         COMMENT "hello A"
>         COMMAND ${CMAKE_COMMAND} -E sleep 3
>         COMMAND ${CMAKE_COMMAND} -E echo "hello A")
>
> add_custom_command(OUTPUT b.custom
>         COMMENT "hello B"
>         COMMAND ${CMAKE_COMMAND} -E touch b.custom)
>
> add_executable(B b.c b.custom)
> add_dependencies(B A)
>
> a.c:
> void foo() {}
>
> b.c:
> int main() { return 0; }
>
> In CMake 3.7:
>
> build cmake_order_depends_target_B: phony || A.dll b.custom
> build b.custom: CUSTOM_COMMAND || A.dll
> build CMakeFiles\B.dir\b.c.obj: C_COMPILER__B C$:\Users\re41236\Desktop\test\cmake\b.c || cmake_order_depends_target_B
> build B.exe: C_EXECUTABLE_LINKER__B CMakeFiles\B.dir\b.c.obj || A.dll
>
> In CMake 3.9:
> build cmake_object_order_depends_target_B: phony || b.custom cmake_object_order_depends_target_A
> build CMakeFiles\B.dir\b.c.obj: C_COMPILER__B C$:\Users\re41236\Desktop\test\cmake\b.c || cmake_object_order_depends_target_B
> build b.custom: CUSTOM_COMMAND || A.dll
> build B.exe: C_EXECUTABLE_LINKER__B CMakeFiles\B.dir\b.c.obj || A.dll
>
> So in 3.7, the add_dependencies(B A) put A.dll as an order-only dependency of all the rules that were part of B; nothing in B builds until everything in A does.
> In 3.9 only the custom commands and link rules get it, the object rules don't. And there doesn't seem to be a way to explicitly get an order-only target dependency into the object rules in the (rare) cases where it actually was needed, like #import. So not only is it a breaking change, it’s not readily fixed.
>
> Admittedly right now this is pretty much limited to weird compiler (mis-)features like MSVC's #import that reference input besides the source/headers. Although it might become more common if the C++ modules TS ever catches on (you'll need the .ifc file generated by compiling the dependency before compiling objects that use it)
>
>> > > When adding
>> > > this custom command, you may use the `BYPRODUCTS` argument
>> > > (introduced in 3.2.0) to let CMake know what's going on here. It
>> > > only affects Ninja, but the other generators do target-level
>> > > dependencies anyways. That output can then be depended on via
>> > > `OBJECT_DEPENDS` and the dependency should link up properly.
>> >
>> > There is not an explicit file output, though I could do the usual
>> > workaround of a stamp/witness file listed in BYPRODUCTS to the
>> > add_custom_command(TARGET ... POST_BUILD ...). But I don't think that
>> > will work with most generators, since CMake doesn't generally allow
>> > file-level depends to set the order in which targets are built. I
>> > suppose it might work out in practice for ninja since that writes a
>> > monolithic set of rules, but conditional code where I have to peek at
>> > CMAKE_GENERATOR and use BYPRODUCTS/OBJECT_DEPENDS for ninja
>> and
>> > add_dependencies for other generators seems like the sort of thing
>> > this list would tell me not to do :-)
>>
>> Well, other generators are generally target-ordered anyways. Ninja is the
>> oddball here (which is why it's the only one to get the feature). I don't know
>> the effect it'd have in other generators, but I feel like I'd be surprised if it
>> *broke* them since excess dependencies (usually) only result in either
>> slower builds or circular dependency loops and Ninja complains loudly about
>> the latter. And since BYPRODUCTS only affects Ninja, if BYPRODUCTS is used,
>> other generators shouldn't care anyways.
>
> True, I suppose I don’t need to peek at CMAKE_GENERATOR, I could just do it both ways all the time. As long as nobody comes up with a generator that can separate compile vs link dependencies, but still can't handle cross-subdir file dependencies.
>
>> > And even for ninja I think I'd have to be making undocumented
>> > assumptions about the binary dir layout to refer to my witness file
>> > that was generated in a different subdir's CMakelists.txt.
>>
>> There's nothing stopping the witness files couldn't all be under a single
>> directory (such as ${CMAKE_BINARY_DIR}/CMakeFiles/tlb).
>
> Ok, good point. They don't have to be alongside the target's other artifacts.
>
>> > > If it instead gets registered somewhere in the aether (as far as
>> > > CMake is concerned), adding support for generator expressions to
>> > > `OBJECT_DEPENDS` so that `$<TARGET_FILE:tgt>` may be used there
>> would be the next solution.
>> >
>> > Yes, the dependency in question for #import is on information
>> > "somewhere in the aether" (or rather the Win32 registry).
>> >
>> > Supporting $<TARGET_FILE> does in OBJECT_DEPENDS would be a great
>> > solution for my first use case of a embedding that file in a resource.
>> > But I don't think that helps with #import, since I don't actually want
>> > to read the $<TARGET_FILE>, I just want the post-build that registers
>> > it to have run.
>>
>> POST_BUILD rules are attached to the target, so depending on the target also
>> guarantees that the POST_BUILD command(s) have run as well.
>
> Right, which is how I did it before. What I was saying (not very clearly) is that I only want the target to have been registered (so it's POST_BUILD has run), I don't care if it's newer. I only care if the .tlb file is newer, and I can do that with OBJECT_DEPENDS already. So the target part ought to be an order-only dependency (as it was in 3.8). Depending on $<TARGET_FILE> is sufficient, but actually too strong.
>
>> > Also, in the cases of .tlb files that are *not* embedded in DLL
>> > resources, the target in question is going to be an add_custom_target
>> > from another subdirectory; the .tlb file is built by an
>> > add_custom_command(OUTPUT...) but this rule gets emitted in an
>> > add_custom_target that depends on this file to build it and then
>> > registers it. If each subdir had the add_custom_command instead of
>> > using an intermediate target, multiple targets would each end up with
>> > their own copy of the rule to build the .tlb file, leading to race
>> > conditions where they all try to build it at once and get file-in-use
>> > errors (they can't just build individual copies, because it has to end
>> > up with a unique key referencing the .tlb path in the win32 regist>
>> > ry).
>>
>> Yeah, there should be just one .tlb rule writer. Usually I handle that by
>> collecting information in global properties and writing a rule at the end to
>> handle all of them.
>
> Yeah. It would be really cool to have an end-of-input callback so include files that define their own commands could get a callback hook to write such "rule at the end" functions :-).
>
>> > You're not currently allowed to use $<TARGET_FILE:x> on UTILITY
>> > targets even if the LOCATION property has been set (it's blocked in
>> > TargetFilesystemArtifact::Evaluate with "Target <x> is not an
>> > executable or library"). Maybe that could be changed as well (which
>> > would be nice), but it seems like if one is adding support for
>> > $<TARGET_FILE> generator expressions in OBJECT_DEPENDS (which
>> implies
>> > supporting the generator context and context->DependTargets), it seems
>> > like you as well go the rest of the way and just treat them completely
>> > the same as the DEPENDS argument to add_custom_command, allowing
>> both
>> > file and target dependencies to be listed in the first place.
>>
>> That sounds like a likely path to follow when supporting genexes in
>> OBJECT_DEPENDS.
>
> This would definitely solve my complaint; I'd be able to put target depends on an object rule in the (rare) case that it really needs them.
>
>> > > Making `POST_BUILD` write out a stamp file would also work and then
>> > > using `OBJECT_DEPENDS` on that would also work.
>> >
>> > No, as above I don't think that would be legal across subdirs, at
>> > least in the context of a CMake file that's supposed to work with
>> > various generators. Feel free to correct me if I'm wrong about that...
>>
>> Experiments would be more useful. add_custom_* have some of the most
>> complicated interaction semantics in CMake. I can't keep all of them straight
>> all the time (usually I rediscover them when necessary; I should probably
>> write up some docs next time I need to do so).
>>
>> --Ben
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Loading...