Call function from name and list, including empty elements?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Call function from name and list, including empty elements?

Jan Wielemaker
Hi

I'm converting a big project (SWI-Prolog) to cmake. Still a newbie wrt.
cmake, but quite happy. There is one thing that I can't get done:
install the system using symbolic links to the source tree. This is in
this case really handy for developers.

I've come as far as understanding:

   - The generator creates cmake_install.cmake
   - make/ninja/... install calls cmake -P cmake_install.cmake
   - This calls file(INSTALL ....)

So, I thought I can

   - Add a file cmake_ln_install.cmake that
     - redefines file()
     - includes cmake_install.cmake

To redefine file(), I tried this to see whether this works:

function(file)
   message("Calling file(${ARGN})")
   _file(${ARGN})
endfunction()

But ... some of the file() calls contain empty strings ("") in the
argument vector.  All the rest works fine.

On stack overflow [1], the suggestion was to use "${ARGN}", which
indeed keeps the empty lists and works fine if the receiver is a
function iterating over its argument vector.  After downloading
the cmake source it turns out file() is builtin (C++) and complains
to require at least 2 arguments.  I tried passing the first two
as file(${arg1} ${arg2} "${ARGN}"), which would (I think) work for
a user function, but doesn't for a builtin.

So, my question is

   - Is there a way to redefine a builtin function that passes
     empty strings correctly to the original?
   - If not, should this be considered a bug ...

        Thanks --- Jan

P.s. I'd also like to see the possibility to create symlinks
        in cmake, other than calling cmake -E ...

[1] https://stackoverflow.com/questions/52480737/pass-empty-strings-in-cmake
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Call function from name and list, including empty elements?

Craig Scott-3


On Sun, Sep 30, 2018 at 2:28 AM Jan Wielemaker <[hidden email]> wrote:
Hi

I'm converting a big project (SWI-Prolog) to cmake. Still a newbie wrt.
cmake, but quite happy. There is one thing that I can't get done:
install the system using symbolic links to the source tree. This is in
this case really handy for developers.

This would be very unusual. An installed project should not generally require that the sources it was built from remain around.

 

I've come as far as understanding:

   - The generator creates cmake_install.cmake
   - make/ninja/... install calls cmake -P cmake_install.cmake
   - This calls file(INSTALL ....)

So, I thought I can

   - Add a file cmake_ln_install.cmake that
     - redefines file()
     - includes cmake_install.cmake

You will likely find it more convenient to use install(CODE) or install(SCRIPT) to define commands to add to the install process rather than trying to work with cmake_install.cmake directly. Those two forms of the install() command are the recommended way to get your own code into the cmake_install.cmake file that CMake generates. At the moment, you would need to use execute_process() to invoke cmake -E create_symlink to create the actual link within that code or script (see further below for state of the file() command for this).

 

To redefine file(), I tried this to see whether this works:

function(file)
   message("Calling file(${ARGN})")
   _file(${ARGN})
endfunction()

Overriding built-in functions is strongly discouraged. Apart from relying on undocumented CMake behavior, the above has the potential to result in infinite recursion. I recently wrote a blog article about this which explains in more detail:



 

But ... some of the file() calls contain empty strings ("") in the
argument vector.  All the rest works fine.

On stack overflow [1], the suggestion was to use "${ARGN}", which
indeed keeps the empty lists and works fine if the receiver is a
function iterating over its argument vector.  After downloading
the cmake source it turns out file() is builtin (C++) and complains
to require at least 2 arguments.  I tried passing the first two
as file(${arg1} ${arg2} "${ARGN}"), which would (I think) work for
a user function, but doesn't for a builtin.

[1] https://stackoverflow.com/questions/52480737/pass-empty-strings-in-cmake 

So, my question is

   - Is there a way to redefine a builtin function that passes
     empty strings correctly to the original?
   - If not, should this be considered a bug ...

Builtin functions should not be redefined, so no, it is not considered a bug. ;)

I have a follow-up "part 2" article to the above one I linked, but it is still in preparation. It discusses other problems with trying to forward arguments using ARGN or ARGV, but some of the observations you've made here and in the linked stackoverflow article may also be relevant. If I get time, I'll try to see if I can incorporate some of the observations and behaviors you've raised.

 
P.s.    I'd also like to see the possibility to create symlinks
        in cmake, other than calling cmake -E ...

There is already a feature request with discussion for that here:


 
--
Craig Scott
Melbourne, Australia


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Call function from name and list, including empty elements?

Jan Wielemaker
Hi Craig,

Thanks for the quick response.

On 30/09/18 01:00, Craig Scott wrote:

>
>
> On Sun, Sep 30, 2018 at 2:28 AM Jan Wielemaker <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi
>
>     I'm converting a big project (SWI-Prolog) to cmake. Still a newbie wrt.
>     cmake, but quite happy. There is one thing that I can't get done:
>     install the system using symbolic links to the source tree. This is in
>     this case really handy for developers.
>
>
> This would be very unusual. An installed project should not generally
> require that the sources it was built from remain around.

I'm not proposing this as a standard feature :) SWI-Prolog is a pretty
reflexive system with a lot of development features in it though. This
makes it very practical to edit/develop installed Prolog libraries. This
cannot be done from the source tree because it is the installation step
that assembles the full environment from mostly independent packages.
Right now (when using cmake) I need to go to the installed tree and copy
the modified file(s) back to the sources. That is both inconvenient and
error prone.

Of course, this is only for developers!

>     I've come as far as understanding:
>
>         - The generator creates cmake_install.cmake
>         - make/ninja/... install calls cmake -P cmake_install.cmake
>         - This calls file(INSTALL ....)
>
>     So, I thought I can
>
>         - Add a file cmake_ln_install.cmake that
>           - redefines file()
>           - includes cmake_install.cmake
>
>
> You will likely find it more convenient to use install(CODE)
> <https://cmake.org/cmake/help/latest/command/install.html#code> or
> install(SCRIPT)
> <https://cmake.org/cmake/help/latest/command/install.html#script> to
> define commands to add to the install process rather than trying to work
> with cmake_install.cmake directly. Those two forms of the install()
> command are the recommended way to get your own code into the
> cmake_install.cmake file that CMake generates. At the moment, you would
> need to use execute_process() to invoke cmake -E create_symlink to
> create the actual link within that code or script (see further below for
> state of the file() command for this).

Ok. I'm already using some install(CODE) for post-copy installation
steps. I find the interface rather clumsy as you typically need to call
external processes and need to get the quotes right :( Good news is that
cmake allows you to hide all this stuff behind a function. So, I guess
your proposal would be to define a function that abstracts away over
installing the Prolog (data) files and uses one of the above if some
option is selected and the usual install(FILES ...) otherwise?

>     To redefine file(), I tried this to see whether this works:
>
>     function(file)
>         message("Calling file(${ARGN})")
>         _file(${ARGN})
>     endfunction()
>
> Overriding built-in functions is strongly discouraged. Apart from
> relying on undocumented CMake behavior, the above has the potential to
> result in infinite recursion. I recently wrote a blog article about this
> which explains in more detail:
>
> https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/

Thanks. In this case I'm not that concerned about future robustness as
it is `developers only' and when it breaks we'll find plan B ... It
doesn't need to work everywhere anytime, just using a current recent
version of cmake on Unix-like systems.

>     But ... some of the file() calls contain empty strings ("") in the
>     argument vector.  All the rest works fine.
>
>     On stack overflow [1], the suggestion was to use "${ARGN}", which
>     indeed keeps the empty lists and works fine if the receiver is a
>     function iterating over its argument vector.  After downloading
>     the cmake source it turns out file() is builtin (C++) and complains
>     to require at least 2 arguments.  I tried passing the first two
>     as file(${arg1} ${arg2} "${ARGN}"), which would (I think) work for
>     a user function, but doesn't for a builtin.
>
>     [1]
>     https://stackoverflow.com/questions/52480737/pass-empty-strings-in-cmake
>
>     So, my question is
>
>         - Is there a way to redefine a builtin function that passes
>           empty strings correctly to the original?
>         - If not, should this be considered a bug ...
>
>
> Builtin functions should not be redefined, so no, it is not considered a
> bug. ;)

:(

> I have a follow-up "part 2" article to the above one I linked, but it is
> still in preparation. It discusses other problems with trying to forward
> arguments using ARGN or ARGV, but some of the observations you've made
> here and in the linked stackoverflow article may also be relevant. If I
> get time, I'll try to see if I can incorporate some of the observations
> and behaviors you've raised.

Is it a plan to have something that reliably does forward arguments?
I needed that in some of my internal abstractions (between my own
functions) and I've seen quite a few questions on how to do that.
For example,

     call_function(name a1 a2 ... list)

where list is a variable name rather than an expanded variable, i.e.,
the last argument is always a variable _name_?

>     P.s.    I'd also like to see the possibility to create symlinks
>              in cmake, other than calling cmake -E ...
>
> There is already a feature request with discussion for that here:
>
> https://gitlab.kitware.com/cmake/cmake/issues/16926

Great!

        Cheers --- Jan

P.s. Examining the source a bit further, it seems that installing
        a file that is in fact a symlink will create a symlink.  Would
        a simple way around be to create a secondary tree next to the
         source tree that copies the directory structure and symlinks
         all the files work?

P.s. Migrating from autoconf/make to cmake was a lot of work
        (about 2 weeks for converting 45 Makefile.in and 37
        configure.ac files), but make the build specification
         a lot easier to follow and much shorter.  Especially
        using the ninja backend, building is quite a bit
         faster as well.
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Call function from name and list, including empty elements?

Craig Scott-3
Sorry for the delay, a few more replies to your latest questions embedded below.


On Sun, Sep 30, 2018 at 6:06 PM Jan Wielemaker <[hidden email]> wrote:

>     I've come as far as understanding:
>
>         - The generator creates cmake_install.cmake
>         - make/ninja/... install calls cmake -P cmake_install.cmake
>         - This calls file(INSTALL ....)
>
>     So, I thought I can
>
>         - Add a file cmake_ln_install.cmake that
>           - redefines file()
>           - includes cmake_install.cmake
>
>
> You will likely find it more convenient to use install(CODE)
> <https://cmake.org/cmake/help/latest/command/install.html#code> or
> install(SCRIPT)
> <https://cmake.org/cmake/help/latest/command/install.html#script> to
> define commands to add to the install process rather than trying to work
> with cmake_install.cmake directly. Those two forms of the install()
> command are the recommended way to get your own code into the
> cmake_install.cmake file that CMake generates. At the moment, you would
> need to use execute_process() to invoke cmake -E create_symlink to
> create the actual link within that code or script (see further below for
> state of the file() command for this).

Ok. I'm already using some install(CODE) for post-copy installation
steps. I find the interface rather clumsy as you typically need to call
external processes and need to get the quotes right :( Good news is that
cmake allows you to hide all this stuff behind a function. So, I guess
your proposal would be to define a function that abstracts away over
installing the Prolog (data) files and uses one of the above if some
option is selected and the usual install(FILES ...) otherwise?

I'm not really making a proposal, but if you can see a robust way to hide away the clumsiness in your particular situation, then go for it.


 

> I have a follow-up "part 2" article to the above one I linked, but it is
> still in preparation. It discusses other problems with trying to forward
> arguments using ARGN or ARGV, but some of the observations you've made
> here and in the linked stackoverflow article may also be relevant. If I
> get time, I'll try to see if I can incorporate some of the observations
> and behaviors you've raised.

Is it a plan to have something that reliably does forward arguments?
I needed that in some of my internal abstractions (between my own
functions) and I've seen quite a few questions on how to do that.
For example,

     call_function(name a1 a2 ... list)

where list is a variable name rather than an expanded variable, i.e.,
the last argument is always a variable _name_?

I'm not aware of any active work going on to more robustly support argument forwarding.


 
P.s.    Examining the source a bit further, it seems that installing
        a file that is in fact a symlink will create a symlink.  Would
        a simple way around be to create a secondary tree next to the
         source tree that copies the directory structure and symlinks
         all the files work?

Erm, maybe try it and see, I don't know off the top of my head. ;)
 

P.s.    Migrating from autoconf/make to cmake was a lot of work
        (about 2 weeks for converting 45 Makefile.in and 37
        configure.ac files), but make the build specification
         a lot easier to follow and much shorter.  Especially
        using the ninja backend, building is quite a bit
         faster as well.

Just wait until you get a chance to add ccache into the mix too. :D

--
Craig Scott
Melbourne, Australia


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake