Managing a local installation of cmake-built open source packages

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Managing a local installation of cmake-built open source packages

ardi
Hi,

I want to install (on UNIX-like systems) a collection of open source
packages which use cmake as the build tool, but I need the
installation to be performed in a local directory (inside my home
directory), and I wish convenient updating to new versions of the
packages.

I didn't arrive to a convincing solution, so any advice will be welcome.

Here are my thoughts:

The trivial solution is of course to directly install to a non-root
prefix when invoking cmake, but, however, this isn't well suited for
updating a previous installation of the packages (building and
installing a new version will only overwrite files that have the same
name, but it will keep old files that no longer exist in the new
version, cluttering the local installation directory with no longer
needed and mismatched files).

A possibility would be to keep a copy of install_manifest.txt whenever
I install a package, and remembering to always run 'xargs rm <
install_manifest.txt' before installing a different version of a
previously installed package.

But keeping the install_manifest.txt of each installed package (and
using it before updating a package) looks like a too-manual task,
candidate to some kind of automation.

Another (perhaps wiser) possibility would be to use cpack for creating
either a RPM or DEB, and then use the corresponding package manager to
install the package. But this has problems too: most package managers
assume a / root installation directory. Also, I use several OSs: OSX,
Linux, and some BSDs, and I'm not sure that either the RPM nor the DEB
pkg managers will work flawlessly across all the OSs I use.

What would you recommend here?

Thanks a lot!
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

Konstantin Podsvirov
Hello Ardi!
 
18.01.2017, 20:39, "ardi" <[hidden email]>:

Hi,

I want to install (on UNIX-like systems) a collection of open source
packages which use cmake as the build tool, but I need the
installation to be performed in a local directory (inside my home
directory), and I wish convenient updating to new versions of the
packages.

I didn't arrive to a convincing solution, so any advice will be welcome.

Here are my thoughts:

The trivial solution is of course to directly install to a non-root
prefix when invoking cmake, but, however, this isn't well suited for
updating a previous installation of the packages (building and
installing a new version will only overwrite files that have the same
name, but it will keep old files that no longer exist in the new
version, cluttering the local installation directory with no longer
needed and mismatched files).

A possibility would be to keep a copy of install_manifest.txt whenever
I install a package, and remembering to always run 'xargs rm <
install_manifest.txt' before installing a different version of a
previously installed package.

But keeping the install_manifest.txt of each installed package (and
using it before updating a package) looks like a too-manual task,
candidate to some kind of automation.

Another (perhaps wiser) possibility would be to use cpack for creating
either a RPM or DEB, and then use the corresponding package manager to
install the package. But this has problems too: most package managers
assume a / root installation directory. Also, I use several OSs: OSX,
Linux, and some BSDs, and I'm not sure that either the RPM nor the DEB
pkg managers will work flawlessly across all the OSs I use.

What would you recommend here?

Thanks a lot!

 
Sounds like, you need cross-platform (several desktops) package manager compatible with cmake.
 
If all desktops with GUI, what about using CPack IFW generator?
 
https://cmake.org/cmake/help/latest/module/CPackIFW.html
 
--
Regards,
Konstantin Podsvirov
 

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

ardi
On Wed, Jan 18, 2017 at 7:19 PM, Konstantin Podsvirov
<[hidden email]> wrote:
> Hello Ardi!
>
[...]
>
> Sounds like, you need cross-platform (several desktops) package manager
> compatible with cmake.
>
> If all desktops with GUI, what about using CPack IFW generator?
>
> https://cmake.org/cmake/help/latest/module/CPackIFW.html

Thanks for the idea, Konstantin. Yes, all the OSs I use have a GUI,
although I don't use Qt. If the best solution is IFW, I'd use Qt only
for this. BTW, I'm new to IFW, and if I'm reading it correctly, cpack
generates an installer, which I guess is an executable... but... can
the installer also be used for uninstalling? Will an installer of a
new version of a package uninstall an older version if it exists?
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

Konstantin Podsvirov
18.01.2017, 21:32, "ardi" <[hidden email]>:

On Wed, Jan 18, 2017 at 7:19 PM, Konstantin Podsvirov
<[hidden email]> wrote:

 Hello Ardi!
 

[...]

 
 Sounds like, you need cross-platform (several desktops) package manager
 compatible with cmake.

 If all desktops with GUI, what about using CPack IFW generator?

 https://cmake.org/cmake/help/latest/module/CPackIFW.html


Thanks for the idea, Konstantin. Yes, all the OSs I use have a GUI,
although I don't use Qt. If the best solution is IFW, I'd use Qt only
for this. BTW, I'm new to IFW, and if I'm reading it correctly, cpack
generates an installer, which I guess is an executable... but... can
the installer also be used for uninstalling? Will an installer of a
new version of a package uninstall an older version if it exists?

CPack IFW generator help you to create installer (yes, it's executable based on Qt,
but static linked) thats you need run on target platform (you don't have preinstalled Qt).
 
CPack IFW generator can also help you to create packages repository to use from installers
(local or from network).
 
Installer will remove old package before update him to new version.
 
--
Regards,
Konstantin Podsvirov
 

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

Guillaume Dumont
You can have a look at:

Hunter: https://github.com/ruslo/hunter
Conan: https://conan.io/

HTH

Guillaume


On Wed, Jan 18, 2017 at 1:47 PM, Konstantin Podsvirov <[hidden email]> wrote:
18.01.2017, 21:32, "ardi" <[hidden email]>:

On Wed, Jan 18, 2017 at 7:19 PM, Konstantin Podsvirov
<[hidden email]> wrote:

 Hello Ardi!
 

[...]

 
 Sounds like, you need cross-platform (several desktops) package manager
 compatible with cmake.

 If all desktops with GUI, what about using CPack IFW generator?

 https://cmake.org/cmake/help/latest/module/CPackIFW.html


Thanks for the idea, Konstantin. Yes, all the OSs I use have a GUI,
although I don't use Qt. If the best solution is IFW, I'd use Qt only
for this. BTW, I'm new to IFW, and if I'm reading it correctly, cpack
generates an installer, which I guess is an executable... but... can
the installer also be used for uninstalling? Will an installer of a
new version of a package uninstall an older version if it exists?

CPack IFW generator help you to create installer (yes, it's executable based on Qt,
but static linked) thats you need run on target platform (you don't have preinstalled Qt).
 
CPack IFW generator can also help you to create packages repository to use from installers
(local or from network).
 
Installer will remove old package before update him to new version.
 
--
Regards,
Konstantin Podsvirov
 

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--
Guillaume Dumont
=========================
[hidden email]

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Managing a local installation of cmake-built open source packages

ardi
Thanks a lot, Guillaume, I didn't know about these tools, it's the first time I hear of hunter and conan.

Thinking twice, I think that what I need is something similar to BSD ports, or MacPorts, but with the difference that I don't want to patch anything, just install and build the source as is, and in a multi build environment.

It seems that both hunter and conan seem to be similar to BSD ports in nature, so I'll look at them.

If anybody has further suggestions, please don't hesitate to tell.

Thanks!


On Wed, Jan 18, 2017 at 9:11 PM, Guillaume Dumont <<a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;dumont.guillaume@gmail.com&#39;)">dumont.guillaume@...> wrote:
> You can have a look at:
>
> Hunter: https://github.com/ruslo/hunter
> Conan: https://conan.io/
>
> HTH
>
> Guillaume
>
>
> On Wed, Jan 18, 2017 at 1:47 PM, Konstantin Podsvirov
> <<a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;konstantin@podsvirov.pro&#39;)">konstantin@...> wrote:
>>
>> 18.01.2017, 21:32, "ardi" <<a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;ardillasdelmonte@gmail.com&#39;)">ardillasdelmonte@...>:
>>
>> On Wed, Jan 18, 2017 at 7:19 PM, Konstantin Podsvirov
>> <<a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;konstantin@podsvirov.pro&#39;)">konstantin@...> wrote:
>>
>>  Hello Ardi!
>>
>>
>> [...]
>>
>>
>>
>>  Sounds like, you need cross-platform (several desktops) package manager
>>  compatible with cmake.
>>
>>  If all desktops with GUI, what about using CPack IFW generator?
>>
>>  https://cmake.org/cmake/help/latest/module/CPackIFW.html
>>
>>
>> Thanks for the idea, Konstantin. Yes, all the OSs I use have a GUI,
>> although I don't use Qt. If the best solution is IFW, I'd use Qt only
>> for this. BTW, I'm new to IFW, and if I'm reading it correctly, cpack
>> generates an installer, which I guess is an executable... but... can
>> the installer also be used for uninstalling? Will an installer of a
>> new version of a package uninstall an older version if it exists?
>>
>> CPack IFW generator help you to create installer (yes, it's executable
>> based on Qt,
>> but static linked) thats you need run on target platform (you don't have
>> preinstalled Qt).
>>
>> CPack IFW generator can also help you to create packages repository to use
>> from installers
>> (local or from network).
>>
>> Installer will remove old package before update him to new version.
>>
>> --
>> Regards,
>> Konstantin Podsvirov
>>
>>
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake
>
>
>
>
> --
> Guillaume Dumont
> =========================
> <a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;dumont.guillaume@gmail.com&#39;)">dumont.guillaume@...


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

Domen Vrankar
2017-01-18 22:37 GMT+01:00 ardi <[hidden email]>:
If anybody has further suggestions, please don't hesitate to tell.

Alternative 1:
I usually go with the platform supported packages even if that means having more than one package format... I prefer less custom dependencies to one-package-to-rule-them-all logic specially since most of the time programs already have to be recompiled on each platform. For me less custom dependencies is worth far more than having only one package type to maintain.

Since you're saying that you can already install the files through CMake you've already won half the battle - the installation part is basis for all CPack packages (you set 2-3 variables and you have minimal RPM or DEB package(s) - even component packages if install commands have components). I don't know about the rest of the packagers but CPackRPM and Deb are quite similar in that regard so there is not too much of a learning curve. If you don't maintain the packages and they don't use CPack it's easy to append the rules at the end of the root CMakeLists.txt (I did that for C++ version of apache qpid a while back and produced minimal component RPM packages for AIX OS in under 5 minutes).

Regarding installation on a different location as I said I'm not too familiar with the rest of the packagers but you can create a custom package database for both RPM and Deb packages and then install them on a different location (relocatable RPM or a bit hackish way with Deb packages - see LONG_FILENAMES/VerifiResult.cmake CMake test for an example with deb packages).

Alternative 2:
You were talking about install_manifest.txt file. CPack  Archive packagier (particularly STGZ - self extracting tar gz - version) could easily be extended to produce such file and check for it on next install (asking you if you'd really like to delete the content or something similar).
 Contributing something like that to CPack would even remove the burden of maintaining a patched cpack version yourself.
Drawbacks are that you loose files database that comes with rpm/deb (nobody will warn you if a file has been changed, was present in two packages etc.) and also the currently-not-supported-by-cpack part (but since it's fairly easy to implement we could probably get it into next CMake release).

Regards,
Domen

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

Elizabeth A. Fischer
In reply to this post by ardi
Ardi,

What you describe is pretty much what Spack does.  I would take a look at it, see if it meets your needs.  Chances are, at least some of the packages you need are already included in Spack:


-- Elizabeth

On Wed, Jan 18, 2017 at 12:39 PM, ardi <[hidden email]> wrote:
Hi,

I want to install (on UNIX-like systems) a collection of open source
packages which use cmake as the build tool, but I need the
installation to be performed in a local directory (inside my home
directory), and I wish convenient updating to new versions of the
packages.

I didn't arrive to a convincing solution, so any advice will be welcome.

Here are my thoughts:

The trivial solution is of course to directly install to a non-root
prefix when invoking cmake, but, however, this isn't well suited for
updating a previous installation of the packages (building and
installing a new version will only overwrite files that have the same
name, but it will keep old files that no longer exist in the new
version, cluttering the local installation directory with no longer
needed and mismatched files).

A possibility would be to keep a copy of install_manifest.txt whenever
I install a package, and remembering to always run 'xargs rm <
install_manifest.txt' before installing a different version of a
previously installed package.

But keeping the install_manifest.txt of each installed package (and
using it before updating a package) looks like a too-manual task,
candidate to some kind of automation.

Another (perhaps wiser) possibility would be to use cpack for creating
either a RPM or DEB, and then use the corresponding package manager to
install the package. But this has problems too: most package managers
assume a / root installation directory. Also, I use several OSs: OSX,
Linux, and some BSDs, and I'm not sure that either the RPM nor the DEB
pkg managers will work flawlessly across all the OSs I use.

What would you recommend here?

Thanks a lot!
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

ardi
Thanks a lot, Elisabeth, Domen, Guillaume, and Konstantin,

I believe spack is the closest to what I need. However, all these
solutions (hunter, conan, spack...) have perhaps their strongest focus
in packaging, dependencies, automatic downloads, etc... while I prefer
to do all such tasks myself. I prefer to not have packages, just
download the source in the original developer provided form, untar it,
and to even build it on my own, following the developer instructions.
In other words, I want to be as little intrusive as possible, keeping
the original distribution file as is. Once it's built, then it's the
install phase what is critical, because a previous version of the
package might need to be uninstalled, or there might even be files
with equal names across different packages, as Domen pointed out.

As I said, I think spack is the closest. However, I feel it tries to
automate too much the build. Yes, it gives you a lot of customization
options, but I'm not sure the complexity is worth the effort.

However, I think I can follow the spack design without using spack:
Install every project on a different prefix. Then just keep on the
environment CMAKE_INSTALL_PREFIX set to a colon separated list of all
prefixes of all installed projects, and that's it.

Uninstalling is trivial: delete the installation directory.
Keeping several versions of the same package is trivial too: just set
the currently used version in CMAKE_INSTALL_PREFIX
Updating is trivial as well: Install new version to a new prefix, and
either keep or delete the installation directory of the previous
version, and update CMAKE_INSTALL_PREFIX accordingly.

Of course spack does all this automatically for you, but it does a lot
more, and, as I said, I'm not sure the added complexity and automation
is worth the effort.

I think that by using this approach, I could reconsider moving to
spack in the future (I'd likely have to install all packages from
scratch if I move to spack later, but my directory hierarchy will end
up being the same, so all the work I do now -writing code and
projects- would be reusable without modification).

Thanks a lot for all your ideas!!


On Thu, Jan 19, 2017 at 12:25 AM, Elizabeth A. Fischer
<[hidden email]> wrote:

> Ardi,
>
> What you describe is pretty much what Spack does.  I would take a look at
> it, see if it meets your needs.  Chances are, at least some of the packages
> you need are already included in Spack:
>
> https://github.com/llnl/spack
>
> -- Elizabeth
>
> On Wed, Jan 18, 2017 at 12:39 PM, ardi <[hidden email]> wrote:
>>
>> Hi,
>>
>> I want to install (on UNIX-like systems) a collection of open source
>> packages which use cmake as the build tool, but I need the
>> installation to be performed in a local directory (inside my home
>> directory), and I wish convenient updating to new versions of the
>> packages.
>>
>> I didn't arrive to a convincing solution, so any advice will be welcome.
>>
>> Here are my thoughts:
>>
>> The trivial solution is of course to directly install to a non-root
>> prefix when invoking cmake, but, however, this isn't well suited for
>> updating a previous installation of the packages (building and
>> installing a new version will only overwrite files that have the same
>> name, but it will keep old files that no longer exist in the new
>> version, cluttering the local installation directory with no longer
>> needed and mismatched files).
>>
>> A possibility would be to keep a copy of install_manifest.txt whenever
>> I install a package, and remembering to always run 'xargs rm <
>> install_manifest.txt' before installing a different version of a
>> previously installed package.
>>
>> But keeping the install_manifest.txt of each installed package (and
>> using it before updating a package) looks like a too-manual task,
>> candidate to some kind of automation.
>>
>> Another (perhaps wiser) possibility would be to use cpack for creating
>> either a RPM or DEB, and then use the corresponding package manager to
>> install the package. But this has problems too: most package managers
>> assume a / root installation directory. Also, I use several OSs: OSX,
>> Linux, and some BSDs, and I'm not sure that either the RPM nor the DEB
>> pkg managers will work flawlessly across all the OSs I use.
>>
>> What would you recommend here?
>>
>> Thanks a lot!
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake
>
>
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

ardi
Typo in my last message: All occurrences of CMAKE_INSTALL_PREFIX were
meant to be CMAKE_PREFIX_PATH instead.


On Thu, Jan 19, 2017 at 2:57 PM, ardi <[hidden email]> wrote:

> Thanks a lot, Elisabeth, Domen, Guillaume, and Konstantin,
>
> I believe spack is the closest to what I need. However, all these
> solutions (hunter, conan, spack...) have perhaps their strongest focus
> in packaging, dependencies, automatic downloads, etc... while I prefer
> to do all such tasks myself. I prefer to not have packages, just
> download the source in the original developer provided form, untar it,
> and to even build it on my own, following the developer instructions.
> In other words, I want to be as little intrusive as possible, keeping
> the original distribution file as is. Once it's built, then it's the
> install phase what is critical, because a previous version of the
> package might need to be uninstalled, or there might even be files
> with equal names across different packages, as Domen pointed out.
>
> As I said, I think spack is the closest. However, I feel it tries to
> automate too much the build. Yes, it gives you a lot of customization
> options, but I'm not sure the complexity is worth the effort.
>
> However, I think I can follow the spack design without using spack:
> Install every project on a different prefix. Then just keep on the
> environment CMAKE_INSTALL_PREFIX set to a colon separated list of all
> prefixes of all installed projects, and that's it.
>
> Uninstalling is trivial: delete the installation directory.
> Keeping several versions of the same package is trivial too: just set
> the currently used version in CMAKE_INSTALL_PREFIX
> Updating is trivial as well: Install new version to a new prefix, and
> either keep or delete the installation directory of the previous
> version, and update CMAKE_INSTALL_PREFIX accordingly.
>
> Of course spack does all this automatically for you, but it does a lot
> more, and, as I said, I'm not sure the added complexity and automation
> is worth the effort.
>
> I think that by using this approach, I could reconsider moving to
> spack in the future (I'd likely have to install all packages from
> scratch if I move to spack later, but my directory hierarchy will end
> up being the same, so all the work I do now -writing code and
> projects- would be reusable without modification).
>
> Thanks a lot for all your ideas!!
>
>
> On Thu, Jan 19, 2017 at 12:25 AM, Elizabeth A. Fischer
> <[hidden email]> wrote:
>> Ardi,
>>
>> What you describe is pretty much what Spack does.  I would take a look at
>> it, see if it meets your needs.  Chances are, at least some of the packages
>> you need are already included in Spack:
>>
>> https://github.com/llnl/spack
>>
>> -- Elizabeth
>>
>> On Wed, Jan 18, 2017 at 12:39 PM, ardi <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> I want to install (on UNIX-like systems) a collection of open source
>>> packages which use cmake as the build tool, but I need the
>>> installation to be performed in a local directory (inside my home
>>> directory), and I wish convenient updating to new versions of the
>>> packages.
>>>
>>> I didn't arrive to a convincing solution, so any advice will be welcome.
>>>
>>> Here are my thoughts:
>>>
>>> The trivial solution is of course to directly install to a non-root
>>> prefix when invoking cmake, but, however, this isn't well suited for
>>> updating a previous installation of the packages (building and
>>> installing a new version will only overwrite files that have the same
>>> name, but it will keep old files that no longer exist in the new
>>> version, cluttering the local installation directory with no longer
>>> needed and mismatched files).
>>>
>>> A possibility would be to keep a copy of install_manifest.txt whenever
>>> I install a package, and remembering to always run 'xargs rm <
>>> install_manifest.txt' before installing a different version of a
>>> previously installed package.
>>>
>>> But keeping the install_manifest.txt of each installed package (and
>>> using it before updating a package) looks like a too-manual task,
>>> candidate to some kind of automation.
>>>
>>> Another (perhaps wiser) possibility would be to use cpack for creating
>>> either a RPM or DEB, and then use the corresponding package manager to
>>> install the package. But this has problems too: most package managers
>>> assume a / root installation directory. Also, I use several OSs: OSX,
>>> Linux, and some BSDs, and I'm not sure that either the RPM nor the DEB
>>> pkg managers will work flawlessly across all the OSs I use.
>>>
>>> What would you recommend here?
>>>
>>> Thanks a lot!
>>> --
>>>
>>> Powered by www.kitware.com
>>>
>>> Please keep messages on-topic and check the CMake FAQ at:
>>> http://www.cmake.org/Wiki/CMake_FAQ
>>>
>>> Kitware offers various services to support the CMake community. For more
>>> information on each offering, please visit:
>>>
>>> CMake Support: http://cmake.org/cmake/help/support.html
>>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/cmake
>>
>>
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Managing a local installation of cmake-built open source packages

Elizabeth A. Fischer
In reply to this post by ardi

Aldi,
  
I believe spack is the closest to what I need. However, all these
solutions (hunter, conan, spack...) have perhaps their strongest focus
in packaging, dependencies, automatic downloads, etc... while I prefer
to do all such tasks myself.  I prefer to not have packages, just
download the source in the original developer provided form, untar it,
and to even build it on my own, following the developer instructions.

That is exactly what Spack does.  A Spack "package" is really just a recipe that automates those tasks.  I think of the Spack recipe as a Python encoding of the English-language build instructions.  In fact, when I want to know how a package is built, I prefer to look at the Spack recipe.  It's more concise than English, and frequently more complete.
 
As I said, I think spack is the closest. However, I feel it tries to
automate too much the build. Yes, it gives you a lot of customization
options, but I'm not sure the complexity is worth the effort.

Spack should work out-of-the-box without customizing things.  Many of the features --- shell integration, environment modules, etc --- are not strictly necessary.

You might be interested in the `spack setup` command, which currently works with CMake-based projects.  It is especially good for projects you're developing, or otherwise already have the source code in your directory.  I use this feature to configure all the projects I'm working on.  To use Spack setup, you do the following:

 1. Download and untar the software yourself, in your favorite location.  (This is especially useful for development versions that don't yet have a tarball).
 2. Run `spack setup`.  This creates a script whose purpose it is to call CMake with appropriate options for dependencies and install location.
 3. Run the Spack-generated script in place of CMake
 4. Build and install yourself.


Here's an example...
$ tar xvfz myproject.tar.gz
$ cd myproject
$ spack setup myproject@develop
$ mkdir build
$ cd build
$ ../spconfig.py ..   # Calls through to CMake
$ make
$ make install
 
I think that by using this approach, I could reconsider moving to
spack in the future (I'd likely have to install all packages from
scratch if I move to spack later, but my directory hierarchy will end
up being the same, so all the work I do now -writing code and
projects- would be reusable without modification).

With an auto-builder, I've found that re-installing everything is no big deal.  I can re-install 100 packages in a few hours of wall time, and no more than a few minutes of my time.

-- Elizabeth


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake