Looking for an explanation: What exactly means "install" in cmake language?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Looking for an explanation: What exactly means "install" in cmake language?

Cornelis Bockemühl
Constantly I am stumbling over the word "install" in the context of cmake scripts - while it is pretty clear that the word cannot mean what nowadays people would understand by that term! But even reading the docs forwards and backwards, studying examples and some generic cmake tutorials I still do hot have a PRECISE idea about what it is actually doing and what the purpose is. Well, I can see everywhere HOW it is done - but it is sometimes an advantage if you also know WHAT!

I know that this question is probably so stupid that I do not really expect anybody to answer it here in the mailing list, but maybe somebody can point me to a resource that explains it in a language that a "mere mortal programmer" can follow?

So far I have a certain kind of understanding, and sometimes it looks like this is correct, but then again I have my certain doubts. I am using install(TARGETS), install(FILES) and install(EXPORT) so far, and the way I am doing it I mostly picked from some general intros and tutorials. And from what I understand, this is what I expect to happen:

install(FILES): some files are copied from some location in the build tree to some installation location
install(EXPORT): some cmake script is written that a "derived" project can call (through find_package) in order to get knowledge about my project
install(TARGET): not really an idea what should happen: also some copying from build tree to installation tree I suppose

Now I also learned that the install step does not happen during a normal build "all" - like "ninja all", but requires an extra "ninja install" step. And also I saw that you can adapt the target of the installation process like this in your project - if you never ever expect the build to be "installed" anywhere else:

set(CMAKE_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/../install CACHE PATH "" FORCE)

With this I would expect to generate an install folder in parallel to the bin, lib64 and other folders in the build tree, collecting there all the "products" of the build for further processing by a derived project.

But what I see is actually - almost nothing!

Well, the install(FILES) actually copies a file, and the install(EXPORT) indeed generates a cmake script, but from the install(TARGET) I see no effect: the install folder remains completely empty! Still I see in the log many things like "Installing: /.../install/bin/<somefile>" - but I never find then <somefile> in that indicated location. But yes, it said "installing", not "copying" - so I am back to my initial question: what the hell is "installing" in cmake language?

So basically my problem is: I cannot judge whether my script is working because I simply do not know what SHOULD be the effect of install(TARGET)!

Ah yes, my build tree is after a "normal build" ("all", not "install") full of cmake_install.cmake files that would probably have something to do with that "installation" process, but what? Am I supposed to call them on any occasion or are they supposed to be called automatically at some moment?

Finally, I managed to get also some find_package() working for my project. The "heart" of this is a <Project>Config.cmake script that I am generating during the project build, and which I can write in whatever way I want. For example, I am reading everywhere that I am supposed to do an install(EXPORT) in order to make that find_package() happen, but since I do not really understand what it is supposed to do, and I also see that I can simply write that Config file in such a way that it picks things from where the build process puts them, I do not really see the point.

In other words: I tend to simply write config scripts for the import that rely on the build tree and simply skip that install step - because getting this to work is so difficult if you never know what is supposed to happen how and at what stage of the process and under which conditions!

But then I see that others are still using the install() steps, so there is probably still some meaning in it...

So IF such a thing exists, like a comprehensive explanation about what "install" actually means and what it is supposed to do at what stage of the process, I would be very happy and thankful!

Best regards,
Cornelis
 
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Looking for an explanation: What exactly means "install" in cmake language?

J Decker
Install steps are done with `cmake --build . --target install`  (or --target INSTALL on some generators)
It's done after the build is complete, if the build step fails, it will not install.  (Install depends on build automatically)

The install steps can also be used by --target package - which you can use a packager to .tar.gz or NSIS(?) Executable or.....  It needs some CPACK instructions set.


On Mon, Oct 7, 2019 at 6:44 AM Cornelis Bockemühl <[hidden email]> wrote:
Constantly I am stumbling over the word "install" in the context of cmake scripts - while it is pretty clear that the word cannot mean what nowadays people would understand by that term! But even reading the docs forwards and backwards, studying examples and some generic cmake tutorials I still do hot have a PRECISE idea about what it is actually doing and what the purpose is. Well, I can see everywhere HOW it is done - but it is sometimes an advantage if you also know WHAT!

I know that this question is probably so stupid that I do not really expect anybody to answer it here in the mailing list, but maybe somebody can point me to a resource that explains it in a language that a "mere mortal programmer" can follow?

So far I have a certain kind of understanding, and sometimes it looks like this is correct, but then again I have my certain doubts. I am using install(TARGETS), install(FILES) and install(EXPORT) so far, and the way I am doing it I mostly picked from some general intros and tutorials. And from what I understand, this is what I expect to happen:

install(FILES): some files are copied from some location in the build tree to some installation location
install(EXPORT): some cmake script is written that a "derived" project can call (through find_package) in order to get knowledge about my project
install(TARGET): not really an idea what should happen: also some copying from build tree to installation tree I suppose

Now I also learned that the install step does not happen during a normal build "all" - like "ninja all", but requires an extra "ninja install" step. And also I saw that you can adapt the target of the installation process like this in your project - if you never ever expect the build to be "installed" anywhere else:

set(CMAKE_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/../install CACHE PATH "" FORCE)

With this I would expect to generate an install folder in parallel to the bin, lib64 and other folders in the build tree, collecting there all the "products" of the build for further processing by a derived project.

But what I see is actually - almost nothing!

Well, the install(FILES) actually copies a file, and the install(EXPORT) indeed generates a cmake script, but from the install(TARGET) I see no effect: the install folder remains completely empty! Still I see in the log many things like "Installing: /.../install/bin/<somefile>" - but I never find then <somefile> in that indicated location. But yes, it said "installing", not "copying" - so I am back to my initial question: what the hell is "installing" in cmake language?

So basically my problem is: I cannot judge whether my script is working because I simply do not know what SHOULD be the effect of install(TARGET)!

Ah yes, my build tree is after a "normal build" ("all", not "install") full of cmake_install.cmake files that would probably have something to do with that "installation" process, but what? Am I supposed to call them on any occasion or are they supposed to be called automatically at some moment?

Finally, I managed to get also some find_package() working for my project. The "heart" of this is a <Project>Config.cmake script that I am generating during the project build, and which I can write in whatever way I want. For example, I am reading everywhere that I am supposed to do an install(EXPORT) in order to make that find_package() happen, but since I do not really understand what it is supposed to do, and I also see that I can simply write that Config file in such a way that it picks things from where the build process puts them, I do not really see the point.

In other words: I tend to simply write config scripts for the import that rely on the build tree and simply skip that install step - because getting this to work is so difficult if you never know what is supposed to happen how and at what stage of the process and under which conditions!

But then I see that others are still using the install() steps, so there is probably still some meaning in it...

So IF such a thing exists, like a comprehensive explanation about what "install" actually means and what it is supposed to do at what stage of the process, I would be very happy and thankful!

Best regards,
Cornelis
  --

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Looking for an explanation: What exactly means "install" in cmake language?

Eric Noulard
In reply to this post by Cornelis Bockemühl

I'll try some answer.

1) When cmake "runs" (both cmake and generation times) you get your build tree configured and populated 
2) At build times all build artefact are constructed *in the build tree* somewhere
3) Then when you *install* you copy files/targetd/... from the build tree down to the "install location".

The fact that CMake install command are "processed" during 1) does not copy anything before actual [generated] install command are executed during 3). 

Le lun. 7 oct. 2019 à 15:44, Cornelis Bockemühl <[hidden email]> a écrit :
Constantly I am stumbling over the word "install" in the context of cmake scripts - while it is pretty clear that the word cannot mean what nowadays people would understand by that term! But even reading the docs forwards and backwards, studying examples and some generic cmake tutorials I still do hot have a PRECISE idea about what it is actually doing and what the purpose is. Well, I can see everywhere HOW it is done - but it is sometimes an advantage if you also know WHAT!

I know that this question is probably so stupid that I do not really expect anybody to answer it here in the mailing list, but maybe somebody can point me to a resource that explains it in a language that a "mere mortal programmer" can follow?

So far I have a certain kind of understanding, and sometimes it looks like this is correct, but then again I have my certain doubts. I am using install(TARGETS), install(FILES) and install(EXPORT) so far, and the way I am doing it I mostly picked from some general intros and tutorials. And from what I understand, this is what I expect to happen:

install(FILES): some files are copied from some location in the build tree to some installation location
install(EXPORT): some cmake script is written that a "derived" project can call (through find_package) in order to get knowledge about my project
install(TARGET): not really an idea what should happen: also some copying from build tree to installation tree I suppose

Now I also learned that the install step does not happen during a normal build "all" - like "ninja all", but requires an extra "ninja install" step. And also I saw that you can adapt the target of the installation process like this in your project - if you never ever expect the build to be "installed" anywhere else:

set(CMAKE_INSTALL_PREFIX ${CMAKE_CURRENT_BINARY_DIR}/../install CACHE PATH "" FORCE)

With this I would expect to generate an install folder in parallel to the bin, lib64 and other folders in the build tree, collecting there all the "products" of the build for further processing by a derived project.

Nope. CMake will be using the declarative install(...) command and CMAKE_INSTALL_xxx variables values *at CMake times* in order to generate scripts
that will be run when you do "ninja install" / "make install".

 
But what I see is actually - almost nothing!

Well, the install(FILES) actually copies a file, and the install(EXPORT) indeed generates a cmake script, but from the install(TARGET) I see no effect: the install folder remains completely empty! Still I see in the log many things like "Installing: /.../install/bin/<somefile>" - but I never find then <somefile> in that indicated location. But yes, it said "installing", not "copying" - so I am back to my initial question: what the hell is "installing" in cmake language?

Copying some build tree artefact to the specified install location.
 
So basically my problem is: I cannot judge whether my script is working because I simply do not know what SHOULD be the effect of install(TARGET)!

Ah yes, my build tree is after a "normal build" ("all", not "install") full of cmake_install.cmake files that would probably have something to do with that "installation" process, but what? Am I supposed to call them on any occasion or are they supposed to be called automatically at some moment?

cmake_install.cmake are cmake scripts that are executed when you do
"ninja install" i.e. when you build the pseudo target "install".
 
Finally, I managed to get also some find_package() working for my project. The "heart" of this is a <Project>Config.cmake script that I am generating during the project build, and which I can write in whatever way I want. For example, I am reading everywhere that I am supposed to do an install(EXPORT) in order to make that find_package() happen, but since I do not really understand what it is supposed to do, and I also see that I can simply write that Config file in such a way that it picks things from where the build process puts them, I do not really see the point.

In other words: I tend to simply write config scripts for the import that rely on the build tree and simply skip that install step - because getting this to work is so difficult if you never know what is supposed to happen how and at what stage of the process and under which conditions!

But then I see that others are still using the install() steps, so there is probably still some meaning in it...

Your build tree contains absolute path that may not be used to prepare a package (RPM, DEB, or any other binary installer) for which the content should usually be relocatable, i.e. may be installed
at another prefix than the one used for your build tree.


So IF such a thing exists, like a comprehensive explanation about what "install" actually means and what it is supposed to do at what stage of the process, I would be very happy and thankful!

Chapter 25 of Professionnal CMake (https://crascit.com/professional-cmake/) contains many things about that.
 

Best regards,
Cornelis
  --

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake


--
Eric

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? Looking for an explanation: What exactly means "install" in cmake language?

Cornelis Bockemühl
Thanks to both you and J Decker: I would say that this is still the part that I understood! So basically the word "install" in cmake language could be replaced by "copy" more or less in common human language - right?

But then, if it is about "installing" a "target", which is libraries in my case, I would expect the shared libraries to be copied - no?

And this is exactly what does not happen - for no obvious reason! Because some days ago it even happened in my project ONCE - and then not any more. But debugging is not easy because since that moment I changed many things, and basically the reason for my question is that I have no clear idea what EXACTLY should happen if I put a

install(TARGETS mylibrary)

into my CMakeLists.txt. Well, like you explained, and like what I also thought I had understood: nothing should happen during the configure and generate runs of cmake, and also not during the "ninja all" build run, but only during the "ninja install". or else cmake --build . --target install (which in turn calls ninja in my case). Indeed I observed that it does a build for "all" first if the initial project is not up to date.

But then it tells me that it is successfully "installing" mylibrary, but I see no shared library appearing in the install tree! Or rather: it happened once, but not any more - and I should find out what is missing...

My current workaround is indeed that I am trying to avoid the install step altogether and build a crazy construction with configure_file stuff in order to get the libraries to the right place - and I know pretty well that this is NOT the way how things should be done properly. But I am afraid I will be ready with this workaround way faster than I will understand what is going on during this miraculous "install" process!
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? Looking for an explanation: What exactly means "install" in cmake language?

J Decker
A note - INSTALL( FILES ) is only good for data files, if you have scripts that have executable permissions using INSTALL( PROGRAMS ) will get execute flag set too.

On Mon, Oct 7, 2019 at 7:49 AM Cornelis Bockemühl <[hidden email]> wrote:
Thanks to both you and J Decker: I would say that this is still the part that I understood! So basically the word "install" in cmake language could be replaced by "copy" more or less in common human language - right?

But then, if it is about "installing" a "target", which is libraries in my case, I would expect the shared libraries to be copied - no?

Targets use  ,... depending on add_executable, or add_library was used( and/or if STATIC/SHARED is specified in add library )
 
And this is exactly what does not happen - for no obvious reason! Because some days ago it even happened in my project ONCE - and then not any more. But debugging is not easy because since that moment I changed many things, and basically the reason for my question is that I have no clear idea what EXACTLY should happen if I put a

install(TARGETS mylibrary)
I usally put it immediately after the thing that added the target... 

into my CMakeLists.txt. Well, like you explained, and like what I also thought I had understood: nothing should happen during the configure and generate runs of cmake, and also not during the "ninja all" build run, but only during the "ninja install". or else cmake --build . --target install (which in turn calls ninja in my case). Indeed I observed that it does a build for "all" first if the initial project is not up to date.
right, it's an 'install' target.. make install ; ninja install, ... etc ya 

But then it tells me that it is successfully "installing" mylibrary, but I see no shared library appearing in the install tree! Or rather: it happened once, but not any more - and I should find out what is missing...
When I have issues like that, `cmake --trace ...`  option gives a good idea of what cmake thinks it's doing...  or maybe ninja V=1 install ?
It is copy-if-different, so if the build re-builds and generates the same output library it won't re-install...

You can set set( CMAKE_INSTALL_MESSAGE "LAZY" )  which only emits messages for things it actually installs.... (just a side note)

But I don't know why it's not going where you think it is ( somewhere in CMAKE_BINARY_DIR IIRC?  or did you use CMAKE_CURRENT_BINARY_DIR which can be deeply in install)  I do often set my install to 'output' which by default is put into the build directory...


My current workaround is indeed that I am trying to avoid the install step altogether and build a crazy construction with configure_file stuff in order to get the libraries to the right place - and I know pretty well that this is NOT the way how things should be done properly. But I am afraid I will be ready with this workaround way faster than I will understand what is going on during this miraculous "install" process! --

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Looking for an explanation: What exactly means "install" in cmake language?

Eric Noulard
In reply to this post by Cornelis Bockemühl


Le lun. 7 oct. 2019 à 16:49, Cornelis Bockemühl <[hidden email]> a écrit :
Thanks to both you and J Decker: I would say that this is still the part that I understood! So basically the word "install" in cmake language could be replaced by "copy" more or less in common human language - right?

Nope I oversimplified.
This is not a bare copy, e.g. the runtime path (a.k.a. RPATH or RUNPATH) is updated as well.

But then, if it is about "installing" a "target", which is libraries in my case, I would expect the shared libraries to be copied - no?

Yes the shared lib should be coped and its RPATH updated.
 
And this is exactly what does not happen - for no obvious reason! Because some days ago it even happened in my project ONCE - and then not any more. But debugging is not easy because since that moment I changed many things, and basically the reason for my question is that I have no clear idea what EXACTLY should happen if I put a

install(TARGETS mylibrary)

into my CMakeLists.txt. Well, like you explained, and like what I also thought I had understood: nothing should happen during the configure and generate runs of cmake, and also not during the "ninja all" build run, but only during the "ninja install". or else cmake --build . --target install (which in turn calls ninja in my case). Indeed I observed that it does a build for "all" first if the initial project is not up to date.

But then it tells me that it is successfully "installing" mylibrary, but I see no shared library appearing in the install tree! Or rather: it happened once, but not any more - and I should find out what is missing...

.
Are building out-of-tree? (Are your source tree and build tree separate dir?)
If so, did you try removing the entire build tree and try again ?

 
May be there is probably some mixup with your install DESTINATION in your install command.

My current workaround is indeed that I am trying to avoid the install step altogether and build a crazy construction with configure_file stuff in order to get the libraries to the right place - and I know pretty well that this is NOT the way how things should be done properly. But I am afraid I will be ready with this workaround way faster than I will understand what is going on during this miraculous "install" process!

Provide us with a stripped down non-working example and we may dig into it.
Otherwise it is very difficult to guess what is happening in your particular case. 


--
Eric

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? Looking for an explanation: What exactly means "install" in cmake language?

Cornelis Bockemühl
>Le lun. 7 oct. 2019 à 16:49, Cornelis Bockemühl <[hidden email]> a écrit :
>
>> Thanks to both you and J Decker: I would say that this is still the part
>> that I understood! So basically the word "install" in cmake language could
>> be replaced by "copy" more or less in common human language - right?
>
>Nope I oversimplified.
>This is not a bare copy, e.g. the runtime path (a.k.a. RPATH or RUNPATH) is
>updated as well.

Well yes, I understood that there are "some" other things happening - and I
did not think about RPATH or RUNPATH - because so far I never cared for them...
At this moment I finally have a setup running where shared libraries are
simply copied over with configure_file - without care for RPATH etc. -,
and it's working finally!

But definitely now like the inventors of cmake have thought it should, with
a lot reinventing wheels etc...

>> But then, if it is about "installing" a "target", which is libraries in my
>> case, I would expect the shared libraries to be copied - no?
>
>Yes the shared lib should be coped and its RPATH updated.
>
>> And this is exactly what does not happen - for no obvious reason! Because
>> some days ago it even happened in my project ONCE - and then not any more.
>> But debugging is not easy because since that moment I changed many things,
>> and basically the reason for my question is that I have no clear idea what
>> EXACTLY should happen if I put a
>>
>> install(TARGETS mylibrary)
>>
>> into my CMakeLists.txt. Well, like you explained, and like what I also
>> thought I had understood: nothing should happen during the configure and
>> generate runs of cmake, and also not during the "ninja all" build run, but
>> only during the "ninja install". or else cmake --build . --target install
>> (which in turn calls ninja in my case). Indeed I observed that it does a
>> build for "all" first if the initial project is not up to date.
>>
>> But then it tells me that it is successfully "installing" mylibrary, but
>> I see no shared library appearing in the install tree! Or rather: it
>> happened once, but not any more - and I should find out what is missing...
>
>Are building out-of-tree? (Are your source tree and build tree separate dir?)
>If so, did you try removing the entire build tree and try again ?

Yes, it's out of tree. And also I did this removal and rebuild!

This is exactly how I realized my problem: I had done once an "install build"
that copied library files to the target - and everything looked find from then
on. Until I did that exercise - and realized that nothing is copied any more!

And because I could not find any plausible reason why this library copying did
not happen any more, I gave this "install" business up for the moment...

>May be there is probably some mixup with your install DESTINATION in your
>install command.

The problem is that I did not even change anything in that part of the project
from the moment when it "accidentally" happened once...

But sure, it is never pure magic: I MUST have done something "wrong"!

>> My current workaround is indeed that I am trying to avoid the install
>> step altogether and build a crazy construction with configure_file stuff
>> in order to get the libraries to the right place - and I know pretty
>> well that this is NOT the way how things should be done properly. But I
>> am afraid I will be ready with this workaround way faster than I will
>> understand what is going on during this miraculous "install" process!
>
>Provide us with a stripped down non-working example and we may dig into it.
>Otherwise it is very difficult to guess what is happening in your particular
>case.

That's exactly my problem: I am working on a ParaView custom project, and
then trying to "derive" another one from the first! So there is nothing
to be easily "stripped down" - it is simply a monster project - and I am
almost sure if I am going to write a simple dummy project, everything will
just work fine...

And this is why I was asking for a more thorough documentation that what
you can find at Kitware where not the processes are explained, but just
"what to do".

So from what I see, if you start a cmake project from scratch, you can start
with simple concepts and add towards more complex ones, with growing
understanding. However, if you start to build on ParaView, you start in
the middle of an already highly complex project setup, with little documentation
than just examples - and the entire process feels a bit like learning
C++ with only the source code of a C++ compiler at hand: certainly possible,
but takes a horrible amount of time... ;-)

Side remark: Regarding cmake, this is also the reason why some days ago
I was asking for certain improved functionalities for debug support, like
a way to simply dump a list of "current targets" (including imported!),
plus a way to dump also all valid "properties" of targets!

Bottom line: Since such a documentation like I would like to find may not
even exist, I think I should be happy that I have a "somehow working"
solution for the moment!
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Looking for an explanation: What exactly means "install" in cmake language?

fdk17
In reply to this post by Eric Noulard

On Mon, Oct 7, 2019, at 11:25 AM, Eric Noulard wrote:


Le lun. 7 oct. 2019 à 16:49, Cornelis Bockemühl <[hidden email]> a écrit :
Thanks to both you and J Decker: I would say that this is still the part that I understood! So basically the word "install" in cmake language could be replaced by "copy" more or less in common human language - right?

Nope I oversimplified.
This is not a bare copy, e.g. the runtime path (a.k.a. RPATH or RUNPATH) is updated as well.


File permissions are also set according to what was specified in install() via PERMISSIONS which can be different from the file permissions of the file in the build directory.
I think I saw that there was even a "make install/strip" feature that also stripped targets during installation.

I think that the word install is used consistently between GNU autotools and CMake:
2. (Computer Science) to transfer (computer software) from a distribution file to a permanent location on disk, and prepare it for its particular environment and application.

It's that "prepare it for..." clause that makes it different from a straight up copy.

I don't know what may be causing your issues.  I do know that at least one person felt that CMake installation is too aggressive in that it is always installing files that it doesn't need to during the installation step.


--
Fred

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? Looking for an explanation: What exactly means "install" in cmake language?

Cornelis Bockemühl
> I think that the word install is used consistently between GNU autotools and CMake:

Right, looks like it comes from the "good old times" when on Linux computers software was "installed" by downloading, then doing a "make" and a "make install" in order to get it running, while nowadays "installation" is normally done by a package manager that tries to resolve dependencies and works with binaries only. And CMake does some tricky things to do an "install" in order to later on "export" and "import" the code, or prepare it for "pack" etc.

> It's that "prepare it for..." clause that makes it different from a straight up copy.

In my case it looks like it "somehow works" for now without other preparations - but I hate this "solution" because it is against all "good cmake use" that would rather deal with targets and properties only, not with variables and directly copying files around, fiddling with shared library extensions (like so or dll) - but if working "the right way" takes so much time to figure out, it's finally a question of time economy that you finally end up with doing it "the hard way".

(...which is of course the opposite of what a build tool should do for you: It should not make project setup more complicated, but more simple! But well - that's the real world ;-) )
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? ?= Looking for an explanation: What exactly means "install" in cmake language

Cornelis Bockemühl
In reply to this post by J Decker
Dear all those wo answered on this thread: many thanks first of all!

When I was starting this thread I was really a bit desperate - because things did not do what I expected, I felt like in the middle of a "cmake jungle" - and I doubted everything that I ever meant to have understood about cmake, mostly regarding that "install" and "export" business...

The first thing I learned was thus one first certainty about the entire install process: "install" is indeed kind of "copy with a plus". Already very good to know to start with!

Then there was the question if I could provide a strip down version of my project - and my problem was that "stripping down Paraview" is not so simple...

However, I went back to one of my "teachers" - which is this web page -, and started to build a really extremely trivial example that includes the following points:

- a first shared library "shlibbi" that has an exported header, uses an internal class and boost::regex - to have something "completely external".
- a second shared library "shlibbu" that depends on "shlibbi"
- and finally an "example" that depends on "shlibbu"

And indeed - it works! At least in the sense that I can

- build and "install" first "shlibbi"
- then build and "install" "shlibbu", importing "shiibbi" with a find_package(shlibbi...) call
- and finally build "example", importing "shlibbu" again with find_package(shlibbu...) and assuming that it would take care of "shlibbi" automatically

And if I start now example, it does what it is supposed to do! At least if I do it inside the QtCreator that I am using as my development environment. And it does so even if the copy process that I would have expected did STILL not happen - like in my initial problem that was drving me mad.

What I thought SHOULD happen is: if I "import" shlibbi into shlibbu, then "installing" shlibbu should also bring libshlibbi.so into the target library directory. And finally, if I "install" also the "example" main project, I would expect also both libshlibbi.so and libshlibbu.so in some location inside the install directory tree of "example".

Instead, all the 3 binaries (shlibbi.so, shlibbu.so and example) remain in their "install" target trees. And "somehow" within QtCreator, the shared libraries are found if I start example. However, if I open a terminal in in the .../example/release/install/bin folder and type ./example there, it only complains about not finding libshlibbu.so.

And if I manually copy libshlibbi.so and libshlibbu.so to that directory, I am STILL getting that same error message: it does not find the two libraries

patchelf --set-rpath '$ORIGIN/.' example
patchelf --set-rpath '$ORIGIN/.' libshlibbu.so

Only if I added a RPATH to both example and libshlibbu.so, the start from the terminal also succeeded in the install target (bin) directory.

So my initially very unspecific (and a bit desperate...) question is now much more precise: How can I get the *.so shared libraries into the same directory as the example (or else in a "parallel" lib64 library) - together with the correct RPATH settings? Because this I also learned from your answers: part of the "copy plus" is handling the RPATH properly!

And this time I also have a stripped down example that does almost nothing (add 3 + 1) in a relatively complicated way (that even includes some regex pattern matching...)

Thanks and regards,
Cornelis
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

ExportExample.zip (393K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? ?= Looking for an explanation: What exactly means "install" in cmake language

Cornelis Bockemühl
The missing link to my "cmake teacher web page": https://pabloariasal.github.io/2018/02/19/its-time-to-do-cmake-right/

Regards, Cornelis
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? ?= Looking for an explanation: What exactly means "install" in cmake language

Cornelis Bockemühl
In reply to this post by Cornelis Bockemühl
One more finding: the "magic" that QtCreator does to start example also without any additional fiddling with the RPATH: it already contains a RUNPATH, and this points to the shared library libshlibbu.so in it's build tree location, not in the installed location - and the same with libshlibbu.so.

And the "install" process indeed handles the RPATH/RUNPATH: it simply removes it!

Meaning: it I want to end up what I initially expected, I have to do two additional things manually (or maybe there is an automatic way??):

1) copy the shared libraries to the "example" install directory tree
2) add some RPATH or RUNPATH setting to all the binaries that need it

Regards, Cornelis
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?= Looking for an explanation: What exactly means "install" in cmake language

Eric Noulard


Le mar. 8 oct. 2019 à 12:52, Cornelis Bockemühl <[hidden email]> a écrit :
One more finding: the "magic" that QtCreator does to start example also without any additional fiddling with the RPATH: it already contains a RUNPATH, and this points to the shared library libshlibbu.so in it's build tree location, not in the installed location - and the same with libshlibbu.so.

And the "install" process indeed handles the RPATH/RUNPATH: it simply removes it!

Meaning: it I want to end up what I initially expected, I have to do two additional things manually (or maybe there is an automatic way??):

1) copy the shared libraries to the "example" install directory tree
2) add some RPATH or RUNPATH setting to all the binaries that need it

You should read how RPATH handling is done with CMake.

As you already noted there is a build RPATH and an install RPATH.
The first is used when building the second replace the first after installation takes place.

--
Eric

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?= Looking for an explanation: What exactly means "install" in cmake language

Cornelis Bockemühl
Thanks for that hint! But for me the RPATH stuff is only a supplement, because in the context of a Paraview based project, most of my shared libs are plugins, and for these PV comes with its own mechanism to find them.

But then all the more important is the question about actually copying the libs - to a specific location where the plugin finding procedure finds them!

Regards, Cornelis


On 08.10.2019 at 13:10, Eric Noulard <[hidden email]> wrote:



Le mar. 8 oct. 2019 à 12:52, Cornelis Bockemühl <[hidden email]> a écrit :
One more finding: the "magic" that QtCreator does to start example also without any additional fiddling with the RPATH: it already contains a RUNPATH, and this points to the shared library libshlibbu.so in it's build tree location, not in the installed location - and the same with libshlibbu.so.

And the "install" process indeed handles the RPATH/RUNPATH: it simply removes it!

Meaning: it I want to end up what I initially expected, I have to do two additional things manually (or maybe there is an automatic way??):

1) copy the shared libraries to the "example" install directory tree
2) add some RPATH or RUNPATH setting to all the binaries that need it

You should read how RPATH handling is done with CMake.

As you already noted there is a build RPATH and an install RPATH.
The first is used when building the second replace the first after installation takes place.

--
Eric


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? ?= Looking for an explanation: What exactly means "install" in cmake language

Cornelis Bockemühl
Actually I implemented yesterday a way how to copy the shared libs - with configure_file() and a lot of frickling around with the lib names myself, like finding out the extension on Linux and Windows, etc.

Now I tried to include the following command in the "shlibbiConfig.cmake.in" file:

install(TARGETS shlibbi::SHLIBbi
    EXPORT
        shlibbi-targets
    LIBRARY DESTINATION
        ${CMAKE_INSTALL_LIBDIR})

With this, I would expect that I am not "copying files", but "installing targets" - so actually a more abstract and powerful level - if it works! Since the generated shlibbiConfig.cmake would finally run in the context of the importing project, i.e. during a find_package() call, it should actually transfer the lib files from that imported target into the CMAKE_INSTALL_LIBDIR of the calling project - so exactly what I need.

However, the result is an error message:
 

install TARGETS given target "shlibbi::SHLIBbi" which does not exist.


Well, it exists, because it is generated in the auto-generated shlibbiTargets.cmake file like this:

add_library(shlibbi::SHLIBbi SHARED IMPORTED)

and that shlibbiTargets.cmake was called inside shlibbiConfig.cmake BEFORE the above install(TARGETS...) call.

Conclusion: "imported targets" are not "fully valid targets", because while I can now refer to that imported target, like with an #include ... in my source code, or with a successful link to the library, but obviously I cannot "install" that target.

So my question can be even more specified now: Is there a way around this "install blockage" that would allow me to do the required transfer of the shared library into the lib folder of the calling project - and then even further also to the caller's calling project? I mean: with the effect of first moving libshlibbi.so to the lib directory of the shlibbu project, and then both the libshlibbi.so and the libshlibbu.so to the example project - of course including the required adaptation of the RPATH

Because that is what I learned: doing the transfer with install() instead of a file copy through configure_file gives me not only the more abstract level of project organization, but also takes care of the RPATH...

Best regards,
Cornelis

Am Dienstag, Oktober 08, 2019 14:07 CEST, cornelis <[hidden email]> schrieb:
 
Thanks for that hint! But for me the RPATH stuff is only a supplement, because in the context of a Paraview based project, most of my shared libs are plugins, and for these PV comes with its own mechanism to find them.
 
But then all the more important is the question about actually copying the libs - to a specific location where the plugin finding procedure finds them!
 
Regards, Cornelis

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ?==?utf-8?q? ?==?utf-8?q? ?= Looking for an explanation: What exactly means "install" in cmake language

Cornelis Bockemühl
After stripping down the question to "can I install an imported target"? - with the purpose to copy the shared libraries also into the lib directory of the importing project - I could actually also use Google and look for an answer that somebody else might have been asked.

And indeed: this seems to be a rather common question!

And the "official answer" seems to be something like "this bug is a feature". It must have been a change request already years ago, and it was refused with different arguments, like:

1) proper RPATH handling is not possible in this case, because CMake cannot know the way how the shared library was built initially, so it is not able to manipulate it.

At first this looks plausible, but then I was thinking that in Linux you always deal with ELF binaries, and there you have tools to manipulate RPATH/RUNPATH easily, and in Windows there is nothing similar - and others I don't know...

2) the target is already installed, so it makes no sense to install it again; it would only be a waste of disk space.

Well, in my case it would make a lot of sense, and also for any kind of packager that might use the install directory it would be much easier if all the stuff is already in one location.

3...) maybe others - I did not follow further.

In any case the conclusion is: It's like it is, and all my confusion came basically from this strange behaviour - that "install" is not always really "install" (but now I know at least when and why!). And the only way around - if I want to avoid the configure_file solution - would be to do some proper install(FILES...) instead - together with a little hacking regarding what the right files actually are.

Best regards, Cornelis
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake