Contribute Find-module to CMake vs Config-file to upstream

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Contribute Find-module to CMake vs Config-file to upstream

Mateusz Loskot
Hi,

I've been recently trying to update/add Find-modules to CMake:
updated FindJPEG, proposed FindODBC and most recently FindLZ4.

Discussion during review of the FindLZ4 [1] ended with some surprising
conclusions which I, as someone who relies on CMake and occacionally
contributes to CMake, don't necessarily agree with.
I'd like to share some of my thoughts.

[1] https://gitlab.kitware.com/cmake/cmake/merge_requests/2087#note_412640

A bit of a extract from the brief discussion [1]:

The FindLZ4 discussion basically ended with suggestion from Brad that,
instead of adding Find-module to CMake, I should approach LZ4 project
and add Config-file to the library itself.
Then I argued taht Config-files are more complex and I don't feel like
asking projects, which I don't maintain myself, to accept the extra complexity.
Finally, Brad replied that CMake doesn't need LZ4 itself and yet for some
reason is expected to know all about it: versions, headers, libraries, etc.
and that it's not scalable to teach CMake about every project in the world.

This opinion does not surprise me and I understand the rationale.
I argue that CMake does need to know about every project in the world,
or every project that CMake users wish to use, directly or indirectly.
Without the knowledge CMake would simply be useless.

I understand CMake maintainers try to shift the responsibility of recognising
"every project in the world" away from the CMake itself as not scalable.
Good approach to avoid need to scale up the maintenance efforts,
but that is a bad news for CMake users.

As a CMake user, I expect to be able to install CMake and get my dependencies
recognised by CMake as CMake's responsibility
I do not want to care if library A which I depend on is actually
CMake-ignorant or not.
I do not want to learn why library A does not support CMake, as often that is
philosopical reason and asking about it ends up with a rough responses.
(Imagine, there are libraries on GitHub which maintainers do not accept
addition of ***non-intrusive*** single .travis.yml file, because they
don't trust it.)
If a library is strictly GNU Autoconf, I'm on lost position trying to convince
maitnainers to accept CMake stuff and even if they do, they won't be willing to
maintain it - broken Config-files may be deployed with new packages.
That puts users of CMake and such library on lost position:
CMake says no to Find-module, the library does not care either.

As CMake user I don't want to be left to the discretion of package maintainers.
Packages may ignore CMake Config-file existence.
Packages of libraries which provide CMake and alternative build configurations
may not deploy Config-files or Find-modules.

IMHO, CMake should encourage contributions of new Find-modules.

It does not contradict with the recommendation that Config-files
should be preferred.
Once a library that used to be searchable only via Find-module only receives
Config-file, both approaches still can be available, no reason to
remove the Find-module.
The CMake's policy of preferring Config-file can still apply, there is no clash,
it is just users now have choice.
If a Find-module becomes outdated, it doesn't break anything else in CMake
ecosystem but can be spotted and potential contributor may arrive with a fix.

I want to contribute Find-module into a **central place** where I can easily
access it as well as monitor its state and submit fixes if necessary.
From a contributor POV, that is much more effective than jumping between
variety of issue trackers, creating new accounts for one time use or even
sending patches via e-mails to maitnainers - not everything lives in GitHub yet.

Since CMake is still de-facto a standard for building C++ software,
from end-user POV the current situation feels almost like vendor lock-in.

I call CMake to accept *any* Find-module, filtering only based on
quality of CMake
idiomatic scripting.

If CMake does not want Find-* modules within the source tree, then
- set up https://gitlab.kitware.com/cmake/find-modules
- move all existing Find-modules in there
- relax maintenance rules to accept *any* Find-module, provided
--- CMake scripting is decent quality (e.g no community downvotes or rejecting
     reviews for period of quarantine)
--- CI passing tests
- finally, include the complete find-modules archive in the installer
so it is deployed
   aside of CMake itself

If CMake does not care about Find-modules, it should not care or
should care about
them equally - thus presence of such find-modules archive deployed should
not affect the health of CMake installation and its pursue fo
rFind-modules ignorance.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Mateusz Loskot
http://mateusz.loskot.net
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Jorge Perez
+1 to the gitlab central repository

On Mon, 21 May 2018 at 19:39, Mateusz Loskot <[hidden email]> wrote:
Hi,

I've been recently trying to update/add Find-modules to CMake:
updated FindJPEG, proposed FindODBC and most recently FindLZ4.

Discussion during review of the FindLZ4 [1] ended with some surprising
conclusions which I, as someone who relies on CMake and occacionally
contributes to CMake, don't necessarily agree with.
I'd like to share some of my thoughts.

[1] https://gitlab.kitware.com/cmake/cmake/merge_requests/2087#note_412640

A bit of a extract from the brief discussion [1]:

The FindLZ4 discussion basically ended with suggestion from Brad that,
instead of adding Find-module to CMake, I should approach LZ4 project
and add Config-file to the library itself.
Then I argued taht Config-files are more complex and I don't feel like
asking projects, which I don't maintain myself, to accept the extra complexity.
Finally, Brad replied that CMake doesn't need LZ4 itself and yet for some
reason is expected to know all about it: versions, headers, libraries, etc.
and that it's not scalable to teach CMake about every project in the world.

This opinion does not surprise me and I understand the rationale.
I argue that CMake does need to know about every project in the world,
or every project that CMake users wish to use, directly or indirectly.
Without the knowledge CMake would simply be useless.

I understand CMake maintainers try to shift the responsibility of recognising
"every project in the world" away from the CMake itself as not scalable.
Good approach to avoid need to scale up the maintenance efforts,
but that is a bad news for CMake users.

As a CMake user, I expect to be able to install CMake and get my dependencies
recognised by CMake as CMake's responsibility
I do not want to care if library A which I depend on is actually
CMake-ignorant or not.
I do not want to learn why library A does not support CMake, as often that is
philosopical reason and asking about it ends up with a rough responses.
(Imagine, there are libraries on GitHub which maintainers do not accept
addition of ***non-intrusive*** single .travis.yml file, because they
don't trust it.)
If a library is strictly GNU Autoconf, I'm on lost position trying to convince
maitnainers to accept CMake stuff and even if they do, they won't be willing to
maintain it - broken Config-files may be deployed with new packages.
That puts users of CMake and such library on lost position:
CMake says no to Find-module, the library does not care either.

As CMake user I don't want to be left to the discretion of package maintainers.
Packages may ignore CMake Config-file existence.
Packages of libraries which provide CMake and alternative build configurations
may not deploy Config-files or Find-modules.

IMHO, CMake should encourage contributions of new Find-modules.

It does not contradict with the recommendation that Config-files
should be preferred.
Once a library that used to be searchable only via Find-module only receives
Config-file, both approaches still can be available, no reason to
remove the Find-module.
The CMake's policy of preferring Config-file can still apply, there is no clash,
it is just users now have choice.
If a Find-module becomes outdated, it doesn't break anything else in CMake
ecosystem but can be spotted and potential contributor may arrive with a fix.

I want to contribute Find-module into a **central place** where I can easily
access it as well as monitor its state and submit fixes if necessary.
From a contributor POV, that is much more effective than jumping between
variety of issue trackers, creating new accounts for one time use or even
sending patches via e-mails to maitnainers - not everything lives in GitHub yet.

Since CMake is still de-facto a standard for building C++ software,
from end-user POV the current situation feels almost like vendor lock-in.

I call CMake to accept *any* Find-module, filtering only based on
quality of CMake
idiomatic scripting.

If CMake does not want Find-* modules within the source tree, then
- set up https://gitlab.kitware.com/cmake/find-modules
- move all existing Find-modules in there
- relax maintenance rules to accept *any* Find-module, provided
--- CMake scripting is decent quality (e.g no community downvotes or rejecting
     reviews for period of quarantine)
--- CI passing tests
- finally, include the complete find-modules archive in the installer
so it is deployed
   aside of CMake itself

If CMake does not care about Find-modules, it should not care or
should care about
them equally - thus presence of such find-modules archive deployed should
not affect the health of CMake installation and its pursue fo
rFind-modules ignorance.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Johannes Zarl-Zierl
In reply to this post by Mateusz Loskot
Hi Mateusz,

On Montag, 21. Mai 2018 19:39:16 CEST Mateusz Loskot wrote:
> The FindLZ4 discussion basically ended with suggestion from Brad that,
> instead of adding Find-module to CMake, I should approach LZ4 project
> and add Config-file to the library itself.
> Then I argued taht Config-files are more complex and I don't feel like
> asking projects, which I don't maintain myself, to accept the extra
> complexity

There's more that we (as CMake community) could do to make it easier for non-
cmake projects to add a config file, and I think this is a worthwhile goal.

Currently, if the maintainer of an autotools-based project is willing to add a
cmake config file, he or she basically needs to be an expert on cmake. There's
no official template or guide on how to create a cmake config file without relying
on a cmake module to do the work.

If there was a good guide, or maybe even some tooling, cmake config files would
not be much more work for the maintainer as pkg-config files.

> I call CMake to accept *any* Find-module, filtering only based on
> quality of CMake
> idiomatic scripting.

"Filtering based on quality" being the important part.

> If CMake does not want Find-* modules within the source tree, then
> - set up https://gitlab.kitware.com/cmake/find-modules
> - move all existing Find-modules in there
> - relax maintenance rules to accept *any* Find-module, provided
> --- CMake scripting is decent quality (e.g no community downvotes or
> rejecting reviews for period of quarantine)
> --- CI passing tests
> - finally, include the complete find-modules archive in the installer
> so it is deployed
>    aside of CMake itself

That seems like a reasonable way to deal with the situation. Maybe this would
also encourage better quality of the (current) core find-modules. If you take a
look at those, they are in wildly different shape regarding their quality.

> If CMake does not care about Find-modules, it should not care or
> should care about
> them equally

ACK.

Cheers,
  Johannes

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

rleigh
In reply to this post by Mateusz Loskot
On 2018-05-21 18:39, Mateusz Loskot wrote:
> I've been recently trying to update/add Find-modules to CMake:
> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
[…]

> The FindLZ4 discussion basically ended with suggestion from Brad that,
> instead of adding Find-module to CMake, I should approach LZ4 project
> and add Config-file to the library itself.
> Then I argued taht Config-files are more complex and I don't feel like
> asking projects, which I don't maintain myself, to accept the extra
> complexity.
> Finally, Brad replied that CMake doesn't need LZ4 itself and yet for
> some
> reason is expected to know all about it: versions, headers, libraries,
> etc.
> and that it's not scalable to teach CMake about every project in the
> world.

It really doesn't scale.  I've contributed and provided ongoing fixes
for a few Find modules out of practical necessity, for projects which
are effectively incapable of adding a Config-file, either because the
project was stagnant with no new releases (Xerces, Xalan), or where the
project was so complex that it was effectively impossible to add
(Boost).  However, that work should have been done by the upstream.  The
ongoing maintenance cost has been moved from the upstream development,
where it belongs, to a volunteer (me).  It's work which I do because it
provides value to me and my employer so I can use CMake with the
projects I work on, but is otherwise unjustifiable.  You simply can't
expect every Find module to keep pace with upstream development without
a small army of volunteers to keep them all up to date.  If they are
provided with the upstream releases, they are de facto always up to date
because they are directly matched to the library.

Note that for several of these projects (e.g. Xerces, libtiff), I have
actually provided upstream CMake build systems which install
Config-files, so in the future the Find modules can be deprecated and
users can move to the Config-file as these releases make their way into
distributions.  That's semi-altruistic.  I've spent a lot of effort up
front to make CMake infrastructure in these projects maintainable for
the long term, and something which can be delegated to others for
ongoing maintenance.

> This opinion does not surprise me and I understand the rationale.
> I argue that CMake does need to know about every project in the world,
> or every project that CMake users wish to use, directly or indirectly.
> Without the knowledge CMake would simply be useless.

Not useless, that's going a bit far.  It's requiring a little bit of
extra work to be functional.  For libraries without Find modules, you
can usually write your own in a couple of dozen lines.  At that point
your options are:

- to upstream it as a Config-file template to the upstream developers
- to maintain it independently in your project
- to upstream it to cmake.git

Depending upon the library in question, I've done all three at various
times.

> As a CMake user, I expect to be able to install CMake and get my
> dependencies
> recognised by CMake as CMake's responsibility

> If a library is strictly GNU Autoconf, I'm on lost position trying to
> convince
> maitnainers to accept CMake stuff and even if they do, they won't be
> willing to
> maintain it - broken Config-files may be deployed with new packages.
> That puts users of CMake and such library on lost position:
> CMake says no to Find-module, the library does not care either.

If it's using Autoconf, then it can still provide and install CMake
configuration files.  It can use @var@ substitutions in the
Config-file.in template and then install it.  Just like for pkg-config
templates.

Individual projects may have their own policies, and some might not want
to have CMake support.  In general though, if you were to provide a
tested patch that creates the Config-file in AC_OUTPUT and then install
as e.g. cmake_pkgconfig_DATA in Makefile.am, add a distcheck hook and/or
CI logic to test it, and most projects are in my experience willing to
accept the patch.  The only way to find out is to create and submit the
patch, and have a conversation with them about the cost/benefit of
including it.  Some might be worried about the maintenance burden, so
explain the value of it and maybe commit to maintaining it as required.  
Since you'd likely be maintaining it /anyway/, it's not a big task.

> As CMake user I don't want to be left to the discretion of package
> maintainers.
> Packages may ignore CMake Config-file existence.
> Packages of libraries which provide CMake and alternative build
> configurations
> may not deploy Config-files or Find-modules.
>
> IMHO, CMake should encourage contributions of new Find-modules.

As with all things, in my opinion this really comes down to these
questions: "who is going to do the work", and "who is paying for their
time".  Contribution also has implied ongoing maintenance obligations.  
If Find modules are contributed and then not maintained, they will
become broken over time, and who will be expected to keep them updated?  
The CMake core maintainers?  The submitter?  Users who need it?  If it's
upstreamed, it's much less of a liability.  For the stuff I've
contributed, it's mostly been done on my paid work time because it has
value for the work projects, enabling easy building and testing on a
large range of platforms for both our developers and end users.  But
when I move onto new projects or a new job, I'll find it difficult to
justify doing this in my spare time, as much as I might like to.  If you
take a look at the bug list for e.g. FindBoost, there are a lot of open
issues, but it's so huge and complex that I don't have the resources
(different systems) to address all of the defects--that requires
contributions from the people who can actually test these systems I
can't.  The huge complexity here squarely belongs upstream.  FindBoost
is one of the more complex modules, and I can fully sympathise with the
desire to avoid more of these maintenance nightmares.  It's already a
large burden upon both people like me, and I suspect also the CMake core
maintainers in creating, testing and reviewing changes for every Boost
release and for every platform it's expected to run on.  Multiply that
by the total number of Find modules (approximately 150 at present).

One thing I do think could be improved is the behaviour of find_package
and MODULE|NO_MODULE|CONFIG.  In order to allow more transparent
migration from Find modules to Config-files it would be nice if in the
absence of any of these flags, it would firstly try the Config-file and
if it was missing, then fall back to a Find module.  This might have
some compatibility concerns.  However, if the long term goal is to move
away from Find modules, then a means of transitioning like this would be
desirable.  Otherwise, doing this manually in the Find module itself
might be required.


Regards,
Roger
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Brad King
In reply to this post by Johannes Zarl-Zierl
On 05/22/2018 06:27 AM, Johannes Zarl-Zierl wrote:
> There's more that we (as CMake community) could do to make it easier for non-
> cmake projects to add a config file...
[snip]
> not be much more work for the maintainer as pkg-config files.

There is a proposal here:

  https://mwoehlke.github.io/cps/

for a new package specification format to supersede both CMake
package configuration files and pkg-config .pc files with something
sufficient for both use cases.  The "History" section documents
the limitations of both existing approaches and why neither is
sufficient on its own.

Both pkg-config and CMake's find_package command could use the
proposed `.cps` files.  Projects would only have to provide one
kind of spec file that is buildsystem agnostic.

-Brad
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Sergei Nikulov
вт, 22 мая 2018 г. в 18:13, Brad King <[hidden email]>:

> On 05/22/2018 06:27 AM, Johannes Zarl-Zierl wrote:
> > There's more that we (as CMake community) could do to make it easier
for non-
> > cmake projects to add a config file...
> [snip]
> > not be much more work for the maintainer as pkg-config files.

> There is a proposal here:

>    https://mwoehlke.github.io/cps/

> for a new package specification format to supersede both CMake
> package configuration files and pkg-config .pc files with something
> sufficient for both use cases.  The "History" section documents
> the limitations of both existing approaches and why neither is
> sufficient on its own.

> Both pkg-config and CMake's find_package command could use the
> proposed `.cps` files.  Projects would only have to provide one
> kind of spec file that is buildsystem agnostic.


Hello, Brad.

When will CMake with cps support be available?

> -Brad
> --


--
Best Regards,
Sergei Nikulov
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Brad King
On 05/22/2018 11:21 AM, Sergei Nikulov wrote:
>>    https://mwoehlke.github.io/cps/
>
> When will CMake with cps support be available?

The work is still in the design phase of the CPS format itself.
I've added its author in Cc.

-Brad
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Mateusz Loskot
In reply to this post by Johannes Zarl-Zierl
On 22 May 2018 at 12:27, Johannes Zarl-Zierl <[hidden email]> wrote:

> On Montag, 21. Mai 2018 19:39:16 CEST Mateusz Loskot wrote:
>> The FindLZ4 discussion basically ended with suggestion from Brad that,
>> instead of adding Find-module to CMake, I should approach LZ4 project
>> and add Config-file to the library itself.
>> Then I argued taht Config-files are more complex and I don't feel like
>> asking projects, which I don't maintain myself, to accept the extra
>> complexity
>
> There's more that we (as CMake community) could do to make it easier for non-
> cmake projects to add a config file, and I think this is a worthwhile goal.
>
> Currently, if the maintainer of an autotools-based project is willing to add a
> cmake config file, he or she basically needs to be an expert on cmake. There's
> no official template or guide on how to create a cmake config file without relying
> on a cmake module to do the work.

Hi Johannes,

I can onl agree.
In fact, I called the Config-files approach complex partially due to lack of
easy to follow tutorial.
Partially due to beign too lazy to figure step-by-step procedure from
https://cmake.org/cmake/help/v3.11/manual/cmake-packages.7.html

> If there was a good guide, or maybe even some tooling, cmake config files would
> not be much more work for the maintainer as pkg-config files.

This should be no-brainer.
This should be no-brainer to such extend that for cases of canonical
configurations
CMake should generate all the required *-{config|version}.cmake
And, the CMake documentation should tell what to do, how to fix my
CMakeLists.txt
to make help CMake generate all the files.

>> I call CMake to accept *any* Find-module, filtering only based on
>> quality of CMake
>> idiomatic scripting.
>
> "Filtering based on quality" being the important part.

Idiomatic quality of CMake scripts is very important.
I've seen enough cases of CMake scripting abuse to finally
strive for some linting reflex and must-obey conentions.

Very recently, I've proposed FindODBC
https://gitlab.kitware.com/cmake/cmake/merge_requests/2069
A trivial script one may think, but look at number of review-refine iterations
it too me to improve it (acceptance is still pending).

Here, I would like to thank Brad King a ton.
He's patience to review, suggest where and how to improve
every tiniest detail is priceless.
Have a look at the discussion comments for the MR 2069,
learning outcome from that MR alone makes a substantial
CMake coding guide to me.
I do realise such help consumes a lot of Brad's time  and
I do realise it might be one of reasons CMake may prefer to
avoid such heavy engagement of maintainers by 'outsource'
libraries look-up responsibility.
But, perhaps an easier solution would be to move things and relax requiremnets.

    Il meglio è nemico del bene

I wish there was an official CMake coding guide.
(I'm hoping to collect my experience based on reviews of my
MR-s to FindJPEG, FindODBC and FindLZ4).

>> If CMake does not want Find-* modules within the source tree, then
>> - set up https://gitlab.kitware.com/cmake/find-modules
>> - move all existing Find-modules in there
>> - relax maintenance rules to accept *any* Find-module, provided
>> --- CMake scripting is decent quality (e.g no community downvotes or
>> rejecting reviews for period of quarantine)
>> --- CI passing tests
>> - finally, include the complete find-modules archive in the installer
>> so it is deployed
>>    aside of CMake itself
>
> That seems like a reasonable way to deal with the situation. Maybe this would
> also encourage better quality of the (current) core find-modules. If you take a
> look at those, they are in wildly different shape regarding their quality.


Yes, I have noticed the low quality modules.
I am taking baby steps to improve them, at least in (library) areas of
my interest.

My initial story started with the simple premise:
- Find-modules are very useful "guessers"
- Find-modules are going to stay as they complement preferred Config-files
- CMake installs selection of Find-modules which should be allowed to expand
- CMake installs Find-modules of good and bad or broken quality
- CMake does not suffer directly from hosting bad quality Find-modules
- CMake should let the community propose and maintain Find-modules
- CMake should not care if a Find-module becomes outdated as it would
not suffer directly (as per previous points)
...


Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Mateusz Loskot
http://mateusz.loskot.net
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Mateusz Loskot
In reply to this post by rleigh
On 22 May 2018 at 13:36,  <[hidden email]> wrote:

> On 2018-05-21 18:39, Mateusz Loskot wrote:
>>
>> I've been recently trying to update/add Find-modules to CMake:
>> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
> […]
>>
>> The FindLZ4 discussion basically ended with suggestion from Brad that,
>> instead of adding Find-module to CMake, I should approach LZ4 project
>> and add Config-file to the library itself.
>> Then I argued taht Config-files are more complex and I don't feel like
>> asking projects, which I don't maintain myself, to accept the extra
>> complexity.
>> Finally, Brad replied that CMake doesn't need LZ4 itself and yet for some
>> reason is expected to know all about it: versions, headers, libraries,
>> etc.
>> and that it's not scalable to teach CMake about every project in the
>> world.
>
>
> It really doesn't scale.  I've contributed and provided ongoing fixes for a
> few Find modules out of practical necessity, for projects which are
> effectively incapable of adding a Config-file, either because the project
> was stagnant with no new releases (Xerces, Xalan), or where the project was
> so complex that it was effectively impossible to add (Boost).

It does not matter if it scales.
If there is enough interest within the community, Find-modules should be
allowed and maintained. If they are not, they will stagnate.
That will be the community choice.
Still, it will not hurt CMake principles which already does not care.

> However, that work should have been done by the upstream.
> The ongoing maintenance cost
> has been moved from the upstream development, where it belongs, to a
> volunteer (me).  It's work which I do because it provides value to me and my
> employer so I can use CMake with the projects I work on, but is otherwise
> unjustifiable.

Regarding FindBoost.cmake, being a Boost user, as well as
developer and maintainer of Boost libraries, I already know it is PITA.
I, however, have learned FindBoost.cmake is always step behind
and I have learned how to live with that.
There have been discussions about how Boost could take ove FindBoost.cmake
or provide Config-files, but it always ended with discussions about
(too) clever automatic solutions, Boost.Build integrations, bla bla bla
that literally no one is knowledgable enough or interested to maintain
in long term
- should read as "it's going to implode sooner or later anyway".

To me, FindBoost.cmake works in majority of cases (perhaps not quite
for 1.67 ;))
I know enough CMake scripting to be able to attempt to fix stuff.
Better is enemy of good. I prefer good as good enough to get things done.

Finally, Find-modules are just "guesser", they do not promise marksman-like
operating to find every variant of every version of every library.
Find-modules, however, should be there for most, even if lacking
they can serve as useful basis to develop improved Find-modules,
perhaps contribute it back.
If there is not CMake-provided Find-module, then there is cultivation of
chaos of mostly badly written Find-modules scattered caross GitHub
(thanks the spirits GitHub provides usable search engine).


> You simply can't expect every Find module to keep pace with
> upstream development without a small army of volunteers to keep them all up
> to date.

The point is, I do not expect that.
I have mentioned multiple times, Find-modules should be allowed to
become outdated,
awaiting for intersted volunteers to deliver necessary updates.
But, Find-modules should be there regardless.

> If they are provided with the upstream releases, they are de facto
> always up to date because they are directly matched to the library.

IMHO, that is a false expectation or perception.

> Note that for several of these projects (e.g. Xerces, libtiff), I have
> actually provided upstream CMake build systems which install Config-files,
> so in the future the Find modules can be deprecated and users can move to
> the Config-file as these releases make their way into distributions.


Again, my point is, there is no need for deprecating Find-modules.
CMake can automatically prefer Conig-files if present.
CMake users can conciously prefer one or the other.
Point being, there is choice and the choice is left to both sides,
users as well as maintainers, including Find-modules maintainers.

> That's semi-altruistic.  I've spent a lot of effort up front to make CMake
> infrastructure in these projects maintainable for the long term, and
> something which can be delegated to others for ongoing maintenance.

I do realise the hassle may be huge.
Why not simply shift the way we think about Find-modules
- let them in, without deprecating, and let them remain as fall-back
(regardless if
up-to-date, working or not).

>> This opinion does not surprise me and I understand the rationale.
>> I argue that CMake does need to know about every project in the world,
>> or every project that CMake users wish to use, directly or indirectly.
>> Without the knowledge CMake would simply be useless.
>
>
> Not useless, that's going a bit far.  It's requiring a little bit of extra
> work to be functional.  For libraries without Find modules, you can usually
> write your own in a couple of dozen lines.

Yet another point being, I do not want to write my own, maintain
'cmake' subfolder
in numerous projects of mine, I do not want that.
I buy in to CMake to get it covered by CMake
(even if it means I write a Find-module myself and contribute it to CMake).


> At that point your options are:
>
> - to upstream it as a Config-file template to the upstream developers
> - to maintain it independently in your project
> - to upstream it to cmake.git

I aim to eliminate the 2nd option, but sometimes the 3rd option is not feasible
due to CMake bouncing the responsibility to 1st option.

>> If a library is strictly GNU Autoconf, I'm on lost position trying to
>> convince
>> maitnainers to accept CMake stuff and even if they do, they won't be
>> willing to
>> maintain it - broken Config-files may be deployed with new packages.
>> That puts users of CMake and such library on lost position:
>> CMake says no to Find-module, the library does not care either.
>
>
> If it's using Autoconf, then it can still provide and install CMake
> configuration files.  It can use @var@ substitutions in the Config-file.in
> template and then install it.  Just like for pkg-config templates.

Unless, a religious unwillingless plays an important role.

> Individual projects may have their own policies, and some might not want to
> have CMake support.  In general though, if you were to provide a tested
> patch that creates the Config-file in AC_OUTPUT and then install as e.g.
> cmake_pkgconfig_DATA in Makefile.am, add a distcheck hook and/or CI logic to
> test it, and most projects are in my experience willing to accept the patch.
> The only way to find out is to create and submit the patch, and have a
> conversation with them about the cost/benefit of including it.  Some might
> be worried about the maintenance burden, so explain the value of it and
> maybe commit to maintaining it as required.  Since you'd likely be
> maintaining it /anyway/, it's not a big task.

As I have explained in my initial post, I may not always be
intersested in maintaining
CMake support in N different locations/projects/repositories (for
variety of reasons).
I, however, may be happy to maintain it in the single CMake upstream repository.

>> As CMake user I don't want to be left to the discretion of package
>> maintainers.
>> Packages may ignore CMake Config-file existence.
>> Packages of libraries which provide CMake and alternative build
>> configurations
>> may not deploy Config-files or Find-modules.
>>
>> IMHO, CMake should encourage contributions of new Find-modules.
>
>
> As with all things, in my opinion this really comes down to these questions:
> "who is going to do the work", and "who is paying for their time".
> Contribution also has implied ongoing maintenance obligations.

That is excatly what I'm arguing against.

I have submitted number of m4 modules to GNU Autoconf Archive
and nobody objected (unless a module was technically ill-formed)
or asked me if I'm going to maintain those modules.
In fact, I have not done any maintenance for long time, but I have received
numerous patches from users (which I submitted back to the Archive, of course).


> If Find modules are contributed and then not maintained, they will become broken
> over time, and who will be expected to keep them updated?  The CMake core
> maintainers?  The submitter?  Users who need it?
> If it's upstreamed, it's much less of a liability.

I will repeat myself once again: CMake aleady does not care or at least can not
promise a Find-module is up to date. So, there is no harm of a collection of
*official* CMake modules installed aside of CMake is left to be
maintained by community.

> For the stuff I've contributed, it's mostly been
> done on my paid work time because it has value for the work projects,
> enabling easy building and testing on a large range of platforms for both
> our developers and end users.  But when I move onto new projects or a new
> job, I'll find it difficult to justify doing this in my spare time, as much
> as I might like to.  If you take a look at the bug list for e.g. FindBoost,
> there are a lot of open issues, but it's so huge and complex that I don't
> have the resources (different systems) to address all of the defects--that
> requires contributions from the people who can actually test these systems I
> can't.  The huge complexity here squarely belongs upstream.  FindBoost is
> one of the more complex modules, and I can fully sympathise with the desire
> to avoid more of these maintenance nightmares.  It's already a large burden
> upon both people like me, and I suspect also the CMake core maintainers in
> creating, testing and reviewing changes for every Boost release and for
> every platform it's expected to run on.  Multiply that by the total number
> of Find modules (approximately 150 at present).


A module can naturally become outdated. If there is nobody submitting patches,
there should be no problem and CMake should not suffer at all.
It already does not promise up to date Find-modules.
But, IMHO, existence of FindX.cmake increase chances a contributor
will arrive with updates.
If there is no FindX.cmake, there is nothing to update, because there
is nothing to maintain.


> One thing I do think could be improved is the behaviour of find_package and
> MODULE|NO_MODULE|CONFIG.  In order to allow more transparent migration from
> Find modules to Config-files it would be nice if in the absence of any of
> these flags, it would firstly try the Config-file and if it was missing,
> then fall back to a Find module.


I agree. That is should be a must-have feature.

I would even dare to judge that lack of this feature is one of reasons
CMake maintainers dislike Find-modules :-)


Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Mateusz Loskot
http://mateusz.loskot.net
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

CMake mailing list
In reply to this post by rleigh

> On May 22, 2018, at 6:36 AM, [hidden email] wrote:
>
> On 2018-05-21 18:39, Mateusz Loskot wrote:
>> I've been recently trying to update/add Find-modules to CMake:
>> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
> […]
>> The FindLZ4 discussion basically ended with suggestion from Brad that,
>> instead of adding Find-module to CMake, I should approach LZ4 project
>> and add Config-file to the library itself.
>> Then I argued taht Config-files are more complex and I don't feel like
>> asking projects, which I don't maintain myself, to accept the extra complexity.
>> Finally, Brad replied that CMake doesn't need LZ4 itself and yet for some
>> reason is expected to know all about it: versions, headers, libraries, etc.
>> and that it's not scalable to teach CMake about every project in the world.
>
> It really doesn't scale.  I've contributed and provided ongoing fixes for a few Find modules out of practical necessity, for projects which are effectively incapable of adding a Config-file, either because the project was stagnant with no new releases (Xerces, Xalan), or where the project was so complex that it was effectively impossible to add (Boost).  However, that work should have been done by the upstream.  The ongoing maintenance cost has been moved from the upstream development, where it belongs, to a volunteer (me).  It's work which I do because it provides value to me and my employer so I can use CMake with the projects I work on, but is otherwise unjustifiable.  You simply can't expect every Find module to keep pace with upstream development without a small army of volunteers to keep them all up to date.  If they are provided with the upstream releases, they are de facto always up to date because they are directly matched to the library.
>
> Note that for several of these projects (e.g. Xerces, libtiff), I have actually provided upstream CMake build systems which install Config-files, so in the future the Find modules can be deprecated and users can move to the Config-file as these releases make their way into distributions.  That's semi-altruistic.  I've spent a lot of effort up front to make CMake infrastructure in these projects maintainable for the long term, and something which can be delegated to others for ongoing maintenance.
>
>> This opinion does not surprise me and I understand the rationale.
>> I argue that CMake does need to know about every project in the world,
>> or every project that CMake users wish to use, directly or indirectly.
>> Without the knowledge CMake would simply be useless.
>
> Not useless, that's going a bit far.  It's requiring a little bit of extra work to be functional.  For libraries without Find modules, you can usually write your own in a couple of dozen lines.  At that point your options are:
>
> - to upstream it as a Config-file template to the upstream developers
> - to maintain it independently in your project
> - to upstream it to cmake.git
>
> Depending upon the library in question, I've done all three at various times.
>
>> As a CMake user, I expect to be able to install CMake and get my dependencies
>> recognised by CMake as CMake's responsibility
>
>> If a library is strictly GNU Autoconf, I'm on lost position trying to convince
>> maitnainers to accept CMake stuff and even if they do, they won't be willing to
>> maintain it - broken Config-files may be deployed with new packages.
>> That puts users of CMake and such library on lost position:
>> CMake says no to Find-module, the library does not care either.
>
> If it's using Autoconf, then it can still provide and install CMake configuration files.  It can use @var@ substitutions in the Config-file.in template and then install it.  Just like for pkg-config templates.

If a project already uses pkg-config, a user could just use that for its usage requirements. Even more, why couldn’t cmake support this natively? That is the user can just write something like `find_package(zlib CONF)` to get zlib’s usage requirements. This would simplify when a cmake project wants to generate config cmake which uses a pkg-config file, as we can just write `find_dependency(zlib CONF)`.

Paul

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

CMake mailing list
In reply to this post by Brad King

> On May 22, 2018, at 10:13 AM, Brad King <[hidden email]> wrote:
>
> On 05/22/2018 06:27 AM, Johannes Zarl-Zierl wrote:
>> There's more that we (as CMake community) could do to make it easier for non-
>> cmake projects to add a config file...
> [snip]
>> not be much more work for the maintainer as pkg-config files.
>
> There is a proposal here:
>
>  https://mwoehlke.github.io/cps/
>
> for a new package specification format to supersede both CMake
> package configuration files and pkg-config .pc files with something
> sufficient for both use cases.  The "History" section documents
> the limitations of both existing approaches and why neither is
> sufficient on its own.

Or pkg-config could be extended to fix the issues. A `Replaces:` field could be added to support resolving c++11 and c++14 language flags(although changing c++ languages version is a very bad idea and can cause ABI breakage, which why it should be a property of the toolchain).

Also, the link line could support passing the full path to the library rather than using `-L` and `-l` flags.

>
> Both pkg-config and CMake's find_package command could use the
> proposed `.cps` files.  Projects would only have to provide one
> kind of spec file that is buildsystem agnostic.

In the Boost Cmake Modules, we support generating both pkg-config and cmake config files automatically from the cmake targets:

http://bcm.readthedocs.io/en/latest/src/BCMExport.html#bcm-auto-export
http://bcm.readthedocs.io/en/latest/src/BCMPkgConfig.html#bcm-auto-pkgconfig

Paul

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Johannes Zarl-Zierl
In reply to this post by Brad King
On Dienstag, 22. Mai 2018 11:13:18 CEST Brad King wrote:
> > not be much more work for the maintainer as pkg-config files.
>
> There is a proposal here:
>
>   https://mwoehlke.github.io/cps/

This looks quite promising!
Do you know whether there's a shared interest on the pkg-config and autotools
side?

Even without pkg-config joining the effort, it should be much easier to
convince projects to create a .cps file than some "arcane" cmake-config scripts.

Cheers,
  Johannes
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

David Demelier-2
In reply to this post by Mateusz Loskot
On Mon, 2018-05-21 at 19:39 +0200, Mateusz Loskot wrote:

> Hi,
>
> I've been recently trying to update/add Find-modules to CMake:
> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
>
> Discussion during review of the FindLZ4 [1] ended with some
> surprising
> conclusions which I, as someone who relies on CMake and occacionally
> contributes to CMake, don't necessarily agree with.
> I'd like to share some of my thoughts.
>
> [1] https://gitlab.kitware.com/cmake/cmake/merge_requests/2087#note_4
> 12640
>
> A bit of a extract from the brief discussion [1]:
>
> The FindLZ4 discussion basically ended with suggestion from Brad
> that,
> instead of adding Find-module to CMake, I should approach LZ4 project
> and add Config-file to the library itself.

Yes, that's the way to go. It's not the CMake role to add find module
for every single library that exist on earth.

Packages configuration file are not that hard to write and Find
packages are not necessarily easier to write too. You need to create
imported target, check versions, check components and such. All of this
is managed by configuration files automatically when using appropriate
install+export features.

> I understand CMake maintainers try to shift the responsibility of
> recognising
> "every project in the world" away from the CMake itself as not
> scalable.

If you look at pkg-config from Linux world, when you install pkg-config
you just have the tool that inspects .pc files, it does not come with
all libraries implemented in the world. That exactly what should happen
with CMake too.

> Good approach to avoid need to scale up the maintenance efforts,
> but that is a bad news for CMake users.
>
> As a CMake user, I expect to be able to install CMake and get my
> dependencies
> recognised by CMake as CMake's responsibility
> I do not want to care if library A which I depend on is actually
> CMake-ignorant or not.
> I do not want to learn why library A does not support CMake, as often
> that is
> philosopical reason and asking about it ends up with a rough
> responses.
> (Imagine, there are libraries on GitHub which maintainers do not
> accept
> addition of ***non-intrusive*** single .travis.yml file, because they
> don't trust it.)
> If a library is strictly GNU Autoconf, I'm on lost position trying to
> convince
> maitnainers to accept CMake stuff and even if they do, they won't be
> willing to
> maintain it - broken Config-files may be deployed with new packages.
> That puts users of CMake and such library on lost position:
> CMake says no to Find-module, the library does not care either.
>
> As CMake user I don't want to be left to the discretion of package
> maintainers.
> Packages may ignore CMake Config-file existence.

Try to convince them to incorporate a patch, or if they won't I usually
keep a custom FindModule in my own repo and reuse it if required.

> Packages of libraries which provide CMake and alternative build
> configurations
> may not deploy Config-files or Find-modules.
>
> IMHO, CMake should encourage contributions of new Find-modules.

No.

What if CMake 3.12 has a FindFooBar module shipped in your package
manager. But upstream decided to break everything? Change component
name, change library name? We won't have time to inspect all find
modules and update them *because* upstream decided to change.

> I want to contribute Find-module into a **central place** where I can
> easily
> access it as well as monitor its state and submit fixes if necessary.
> From a contributor POV, that is much more effective than jumping
> between
> variety of issue trackers, creating new accounts for one time use or
> even
> sending patches via e-mails to maitnainers - not everything lives in
> GitHub yet.
>
> Since CMake is still de-facto a standard for building C++ software,
> from end-user POV the current situation feels almost like vendor
> lock-in.
>
> I call CMake to accept *any* Find-module, filtering only based on
> quality of CMake
> idiomatic scripting.
>
> If CMake does not want Find-* modules within the source tree, then
> - set up https://gitlab.kitware.com/cmake/find-modules
> - move all existing Find-modules in there
> - relax maintenance rules to accept *any* Find-module, provided
> --- CMake scripting is decent quality (e.g no community downvotes or
> rejecting
>      reviews for period of quarantine)
> --- CI passing tests
> - finally, include the complete find-modules archive in the installer
> so it is deployed
>    aside of CMake itself

This sounds like more of a reasonable proposal. CMake should by itself
not propose any Find* package anymore. A user-managed repository where
everyone could push their module could be a good idea (just like AUR,
PPA, copr, and such do for distribution packages)


Regards,

--
David
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Mateusz Loskot
On 23 May 2018 at 16:37, David Demelier <[hidden email]> wrote:

> On Mon, 2018-05-21 at 19:39 +0200, Mateusz Loskot wrote:
>>
>> I've been recently trying to update/add Find-modules to CMake:
>> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
>>
>> Discussion during review of the FindLZ4 [1] ended with some
>> surprising
>> conclusions which I, as someone who relies on CMake and occacionally
>> contributes to CMake, don't necessarily agree with.
>> I'd like to share some of my thoughts.
>>
>> [1] https://gitlab.kitware.com/cmake/cmake/merge_requests/2087#note_4
>> 12640
>>
>> A bit of a extract from the brief discussion [1]:
>>
>> The FindLZ4 discussion basically ended with suggestion from Brad
>> that,
>> instead of adding Find-module to CMake, I should approach LZ4 project
>> and add Config-file to the library itself.
>
> Yes, that's the way to go.

I hoped it will draw clear from my earlier thoughts that I consider discussion
*if* CMake should encourage Find-modules as pointless, or at least
off-topic here.
We should be discussing not *if*, but *how* to keep Find-modules,
enourage new ones, fix existing ones and make both Find-modules and
Config-files coexist.

Hence, I'm not going to address any of your points trying to convince me
to give up my position. If I do, we will be making circles forever.

>> Packages of libraries which provide CMake and alternative build
>> configurations
>> may not deploy Config-files or Find-modules.
>>
>> IMHO, CMake should encourage contributions of new Find-modules.
>
> No.

Yes.

> What if CMake 3.12 has a FindFooBar module shipped in your package
> manager. But upstream decided to break everything? Change component
> name, change library name? We won't have time to inspect all find
> modules and update them *because* upstream decided to change.

Nothing. That is already the case and nobody expects Find-modules are
100% solid across CMake X versions of libraries it can recognise.

Find-modules are "guessers" and as such CMake does not give hard promises.
That is already the case. I will repeat the premise points from earlier:

- Find-modules are very useful "guessers"
- CMake installs Find-modules of good and bad or broken quality (that
is happening today!)
- CMake does not suffer directly from hosting bad quality Find-modules
- CMake should not care if a Find-module becomes outdated as it would
not suffer directly (as per previous points)

>> I want to contribute Find-module into a **central place** where I can
>> easily
>> access it as well as monitor its state and submit fixes if necessary.
>> From a contributor POV, that is much more effective than jumping
>> between
>> variety of issue trackers, creating new accounts for one time use or
>> even
>> sending patches via e-mails to maitnainers - not everything lives in
>> GitHub yet.
>>
>> Since CMake is still de-facto a standard for building C++ software,
>> from end-user POV the current situation feels almost like vendor
>> lock-in.
>>
>> I call CMake to accept *any* Find-module, filtering only based on
>> quality of CMake
>> idiomatic scripting.
>>
>> If CMake does not want Find-* modules within the source tree, then
>> - set up https://gitlab.kitware.com/cmake/find-modules
>> - move all existing Find-modules in there
>> - relax maintenance rules to accept *any* Find-module, provided
>> --- CMake scripting is decent quality (e.g no community downvotes or
>> rejecting
>>      reviews for period of quarantine)
>> --- CI passing tests
>> - finally, include the complete find-modules archive in the installer
>> so it is deployed
>>    aside of CMake itself
>
> This sounds like more of a reasonable proposal. CMake should by itself
> not propose any Find* package anymore. A user-managed repository where
> everyone could push their module could be a good idea (just like AUR,
> PPA, copr, and such do for distribution packages)

I'm glad we agree here.

I'd like to see such bundle of Find-modules always installed by CMake.
I'd like to see it hosted on gitlab.kitware.com
I'd like to stick to some form of community-powered dictatorship to
ensure minimum quality.
If there is not enough community to maintain it and keep modules up to
date that is fine
- single high quality but outdated FindX.cmake is worth a ton more
than dozen shitty FindX.cmake dangling on GitHub.
CMake scripting lazyness should be kapt away from
gitlab.kitware.com/cmake-find-modules.


Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Mateusz Loskot
http://mateusz.loskot.net
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Elvis Stansvik
Den ons 23 maj 2018 17:18Mateusz Loskot <[hidden email]> skrev:
On 23 May 2018 at 16:37, David Demelier <[hidden email]> wrote:
> On Mon, 2018-05-21 at 19:39 +0200, Mateusz Loskot wrote:
>>
>> I've been recently trying to update/add Find-modules to CMake:
>> updated FindJPEG, proposed FindODBC and most recently FindLZ4.
>>
>> Discussion during review of the FindLZ4 [1] ended with some
>> surprising
>> conclusions which I, as someone who relies on CMake and occacionally
>> contributes to CMake, don't necessarily agree with.
>> I'd like to share some of my thoughts.
>>
>> [1] https://gitlab.kitware.com/cmake/cmake/merge_requests/2087#note_4
>> 12640
>>
>> A bit of a extract from the brief discussion [1]:
>>
>> The FindLZ4 discussion basically ended with suggestion from Brad
>> that,
>> instead of adding Find-module to CMake, I should approach LZ4 project
>> and add Config-file to the library itself.
>
> Yes, that's the way to go.

I hoped it will draw clear from my earlier thoughts that I consider discussion
*if* CMake should encourage Find-modules as pointless, or at least
off-topic here.
We should be discussing not *if*, but *how* to keep Find-modules,
enourage new ones, fix existing ones and make both Find-modules and
Config-files coexist.

Hence, I'm not going to address any of your points trying to convince me
to give up my position. If I do, we will be making circles forever.

>> Packages of libraries which provide CMake and alternative build
>> configurations
>> may not deploy Config-files or Find-modules.
>>
>> IMHO, CMake should encourage contributions of new Find-modules.
>
> No.

Yes.

As for my part, I'll just have to agree to disagree with this. When I first learned about CMake many years ago, I always thought it strange that it came bundled with a bunch of Find modules (compared to say the pkg-config approach), because it seemed to me such an obviously futile effort to get coverage/keep them up to date.

Maybe the bundled modules served (and continues to serve) a purpose in that they encourage the uptake of CMake. But I think CMake has grown so much by now, and has such leverage (people generally are not opposed to being "CMake compatible"), that efforts are better spent making it easy to write config files, and supporting initiatives like that cps thing.

I also don't agree that it doesn't hurt CMake to have a growing number of outdated find modules. It leads to a lot of bug reports. If it's there, people will expect it to work, and when it doesn't they become disappointed. If it wasn't there from the beginning there's no expectation.

My 2 cents


> What if CMake 3.12 has a FindFooBar module shipped in your package
> manager. But upstream decided to break everything? Change component
> name, change library name? We won't have time to inspect all find
> modules and update them *because* upstream decided to change.

Nothing. That is already the case and nobody expects Find-modules are
100% solid across CMake X versions of libraries it can recognise.

Find-modules are "guessers" and as such CMake does not give hard promises.
That is already the case. I will repeat the premise points from earlier:

- Find-modules are very useful "guessers"
- CMake installs Find-modules of good and bad or broken quality (that
is happening today!)
- CMake does not suffer directly from hosting bad quality Find-modules
- CMake should not care if a Find-module becomes outdated as it would
not suffer directly (as per previous points)

>> I want to contribute Find-module into a **central place** where I can
>> easily
>> access it as well as monitor its state and submit fixes if necessary.
>> From a contributor POV, that is much more effective than jumping
>> between
>> variety of issue trackers, creating new accounts for one time use or
>> even
>> sending patches via e-mails to maitnainers - not everything lives in
>> GitHub yet.
>>
>> Since CMake is still de-facto a standard for building C++ software,
>> from end-user POV the current situation feels almost like vendor
>> lock-in.
>>
>> I call CMake to accept *any* Find-module, filtering only based on
>> quality of CMake
>> idiomatic scripting.
>>
>> If CMake does not want Find-* modules within the source tree, then
>> - set up https://gitlab.kitware.com/cmake/find-modules
>> - move all existing Find-modules in there
>> - relax maintenance rules to accept *any* Find-module, provided
>> --- CMake scripting is decent quality (e.g no community downvotes or
>> rejecting
>>      reviews for period of quarantine)
>> --- CI passing tests
>> - finally, include the complete find-modules archive in the installer
>> so it is deployed
>>    aside of CMake itself
>
> This sounds like more of a reasonable proposal. CMake should by itself
> not propose any Find* package anymore. A user-managed repository where
> everyone could push their module could be a good idea (just like AUR,
> PPA, copr, and such do for distribution packages)

I'm glad we agree here.

I'd like to see such bundle of Find-modules always installed by CMake.
I'd like to see it hosted on gitlab.kitware.com
I'd like to stick to some form of community-powered dictatorship to
ensure minimum quality.
If there is not enough community to maintain it and keep modules up to
date that is fine
- single high quality but outdated FindX.cmake is worth a ton more
than dozen shitty FindX.cmake dangling on GitHub.
CMake scripting lazyness should be kapt away from
gitlab.kitware.com/cmake-find-modules.


Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Mateusz Loskot
On 24 May 2018 at 08:45, Elvis Stansvik <[hidden email]> wrote:

> Den ons 23 maj 2018 17:18Mateusz Loskot <[hidden email]> skrev:
>> On 23 May 2018 at 16:37, David Demelier <[hidden email]> wrote:
>> > On Mon, 2018-05-21 at 19:39 +0200, Mateusz Loskot wrote:
>> >>
>> >>
>> >> IMHO, CMake should encourage contributions of new Find-modules.
>> >
>> > No.
>>
>> Yes.
>
>
> As for my part, I'll just have to agree to disagree with this.
> [...]
> But I think CMake has grown so much by
> now, and has such leverage (people generally are not opposed to being "CMake
> compatible"), that efforts are better spent making it easy to write config
> files, and supporting initiatives like that cps thing.

Those will take years to get into production state for CMake.
Find-modules can solve issues now, without "want to drive? build your
own car first"
appraoch, but  basic CMake scripting knowledge.
I'm very supportive to the cps, but I can not help it move forward.
I, however, can help updating/adding Find-modules.

> I also don't agree that it doesn't hurt CMake to have a growing number of
> outdated find modules. It leads to a lot of bug reports.

In the times of GitHub-like pace of development, we should really learn to use
https://github.com/apps/stale or equivalents.

> If it's there, people will expect it to work, and when it doesn't they become disappointed.
> If it wasn't there from the beginning there's no expectation.

There is nothing wrong with it, as long as status of Find-modules is clearly
stated in the documentation and users learn correctly what Find-modules are
- guessers about future without promises, not configurators about present state.

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
--
Mateusz Loskot
http://mateusz.loskot.net
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

Brad King
In reply to this post by CMake mailing list
On 05/22/2018 10:06 PM, Paul Fultz II wrote:
> Or pkg-config could be extended to fix the issues.

The `.pc` file format is too flat to lend itself well to representing
all the information we need.  A goal of `.cps` files is to teach
pkg-config to parse them and respond to its standard queries with the
same output format as now.

> In the Boost Cmake Modules, we support generating both pkg-config and
> cmake config files automatically from the cmake targets:

That's fantastic but until there is no such thing as a Boost installation
that doesn't support both tools the problems will persist.  They mangle
their library names in such a way that it's impossible to use boost without
a build system that memorizes dozens of mangling conventions that vary
with the version of boost, version of the compiler, architecture, etc.

-Brad
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Contribute Find-module to CMake vs Config-file to upstream

CMake mailing list

> On May 24, 2018, at 8:07 AM, Brad King <[hidden email]> wrote:
>
> On 05/22/2018 10:06 PM, Paul Fultz II wrote:
>> Or pkg-config could be extended to fix the issues.
>
> The `.pc` file format is too flat to lend itself well to representing
> all the information we need.  

What do you mean? What information can’t be represented?


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: pkg-config file format versus CMake packages

Brad King
On 05/24/2018 07:39 PM, Paul Fultz II wrote:
>> On May 24, 2018, at 8:07 AM, Brad King wrote:
>> The `.pc` file format is too flat to lend itself well to representing
>> all the information we need.  
>
> What do you mean? What information can't be represented?

Try running CMake's ExportImport test and take a look at the
files generated by export() and install(EXPORT).  If anyone
can use `.pc` files as a package representation for everything
that test does on all platforms then I'd like to see a proposal.

-Brad
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: pkg-config file format versus CMake packages

CMake mailing list

> On May 25, 2018, at 8:07 AM, Brad King <[hidden email]> wrote:
>
> On 05/24/2018 07:39 PM, Paul Fultz II wrote:
>>> On May 24, 2018, at 8:07 AM, Brad King wrote:
>>> The `.pc` file format is too flat to lend itself well to representing
>>> all the information we need.  
>>
>> What do you mean? What information can't be represented?
>
> Try running CMake's ExportImport test and take a look at the
> files generated by export() and install(EXPORT).  If anyone
> can use `.pc` files as a package representation for everything
> that test does on all platforms then I'd like to see a proposal.

Yes, and what seems to be missing from pkg-config to represent that information is the ability to put a direct path to the library(instead of using the -l and -L flags), and support for a `Replaces` field.

This is also the same shortcomings talked about in the CPS document as well. Resolving these shortcomings, will allow pkg-config to represent the same thing as CPS and cmake. What else is missing?

The reason I say this is that extending pkg-config seems like it would help adoption rather then creating a completely new format. There is already a good portion of open source projects that already support pkg-config, so tweaking them to support more complicated scenarios seems easier than converting everything to a new format.

Paul
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
12