ctest & git submodules

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

ctest & git submodules

Clinton Stimpson

Is there anything special I need to do with ctest so that the ctest_update()
will recognize git submodules and do any init/update of those for me?  I
assumed it would be automatic, but that doesn't seem to be the case.

--
Clinton Stimpson
Elemental Technologies, Inc
Computational Simulation Software, LLC
www.csimsoft.com
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

norulez
Hi,

I don't think so, you could check for a .git directory and then call

git submodule update --init --recursive

Best Regards

Am 26.02.2013 um 18:34 schrieb Clinton Stimpson <[hidden email]>:

>
> Is there anything special I need to do with ctest so that the ctest_update()
> will recognize git submodules and do any init/update of those for me?  I
> assumed it would be automatic, but that doesn't seem to be the case.
>
> --
> Clinton Stimpson
> Elemental Technologies, Inc
> Computational Simulation Software, LLC
> www.csimsoft.com
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

norulez
Here is a working example:

## -- Update git submodules
if (EXISTS "${CTEST_SOURCE_DIRECTORY}/.gitmodules")
message (" -- Update git submodules ${MODEL} - ${CTEST_BUILD_NAME} --")
execute_process (COMMAND ${GIT_EXECUTABLE} submodule update --init --recursive
WORKING_DIRECTORY ${CTEST_SOURCE_DIRECTORY})
endif ()

Best Regards

Am 26.02.2013 um 20:16 schrieb NoRulez <[hidden email]>:

> Hi,
>
> I don't think so, you could check for a .git directory and then call
>
> git submodule update --init --recursive
>
> Best Regards
>
> Am 26.02.2013 um 18:34 schrieb Clinton Stimpson <[hidden email]>:
>
>>
>> Is there anything special I need to do with ctest so that the ctest_update()
>> will recognize git submodules and do any init/update of those for me?  I
>> assumed it would be automatic, but that doesn't seem to be the case.
>>
>> --
>> Clinton Stimpson
>> Elemental Technologies, Inc
>> Computational Simulation Software, LLC
>> www.csimsoft.com
>> --
>>
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Follow this link to subscribe/unsubscribe:
>> http://www.cmake.org/mailman/listinfo/cmake
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Clinton Stimpson

Thanks.  I may have to do that for now.

However, it seems to me that ctest already does a
git submodule update --recurse
but its missing the --init flag to deal with changes to the .gitmodules file.

Clint

On Tuesday, February 26, 2013 08:26:21 PM NoRulez wrote:

> Here is a working example:
>
> ## -- Update git submodules
> if (EXISTS "${CTEST_SOURCE_DIRECTORY}/.gitmodules")
> message (" -- Update git submodules ${MODEL} - ${CTEST_BUILD_NAME} --")
> execute_process (COMMAND ${GIT_EXECUTABLE} submodule update --init
> --recursive WORKING_DIRECTORY ${CTEST_SOURCE_DIRECTORY})
> endif ()
>
> Best Regards
>
> Am 26.02.2013 um 20:16 schrieb NoRulez <[hidden email]>:
> > Hi,
> >
> > I don't think so, you could check for a .git directory and then call
> >
> > git submodule update --init --recursive
> >
> > Best Regards
> >
> > Am 26.02.2013 um 18:34 schrieb Clinton Stimpson <[hidden email]>:
> >> Is there anything special I need to do with ctest so that the
> >> ctest_update() will recognize git submodules and do any init/update of
> >> those for me?  I assumed it would be automatic, but that doesn't seem to
> >> be the case.>
> > --
> >
> > Powered by www.kitware.com
> >
> > Visit other Kitware open-source projects at
> > http://www.kitware.com/opensource/opensource.html
> >
> > Please keep messages on-topic and check the CMake FAQ at:
> > http://www.cmake.org/Wiki/CMake_FAQ
> >
> > Follow this link to subscribe/unsubscribe:
> > http://www.cmake.org/mailman/listinfo/cmake
--
Clinton Stimpson
Elemental Technologies, Inc
Computational Simulation Software, LLC
www.csimsoft.com
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Marcus D. Hanwell-2
On Tue, Feb 26, 2013 at 2:36 PM, Clinton Stimpson <[hidden email]> wrote:
>
> Thanks.  I may have to do that for now.
>
> However, it seems to me that ctest already does a
> git submodule update --recurse
> but its missing the --init flag to deal with changes to the .gitmodules file.
>
It also misses git submodule sync to deal with changes in git
submodule URL, and reset --hard etc. We have just been dealing with a
few of these issues and currently call git directly to sync, init, and
then use submodule foreach to reset and clean all submodules before
updating.

I wonder if any of these are good candidates for adding to ctest in the future.

Marcus
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Jean-Christophe Fillion-Robin
+1 to add these into CTest :) What would be the argument against it ?
Jc


On Tue, Feb 26, 2013 at 2:40 PM, Marcus D. Hanwell <[hidden email]> wrote:
On Tue, Feb 26, 2013 at 2:36 PM, Clinton Stimpson <[hidden email]> wrote:
>
> Thanks.  I may have to do that for now.
>
> However, it seems to me that ctest already does a
> git submodule update --recurse
> but its missing the --init flag to deal with changes to the .gitmodules file.
>
It also misses git submodule sync to deal with changes in git
submodule URL, and reset --hard etc. We have just been dealing with a
few of these issues and currently call git directly to sync, init, and
then use submodule foreach to reset and clean all submodules before
updating.

I wonder if any of these are good candidates for adding to ctest in the future.

Marcus
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake



--
+1 919 869 8849

--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Brad King
On 2/26/2013 2:52 PM, Jean-Christophe Fillion-Robin wrote:
> +1 to add these into CTest :) What would be the argument against it ?

Not every project wants every submodule checked out all the time.
A major use case for them is to have an umbrella project with many
submodules and the developer may only checkout and work on some.
Some may even be proprietary and inaccessible to some machines.

>     > However, it seems to me that ctest already does a
>     > git submodule update --recurse

This is the proper command to update everything that is already
to configured to checkout in the source tree.

>     > but its missing the --init flag to deal with changes to the
>     > .gitmodules file.
>     It also misses git submodule sync to deal with changes in git
>     submodule URL,

These are all intentionally missing for the above reason.  We
should honor the user's configuration.  Maybe they intentionally
use a custom url for a submodule for the branch they test.  We
should not blow away their configuration by default.

> and reset --hard etc.

We do a reset --hard at the top level but I do not think we
do it in the submodules.  That may be worth adding, perhaps
with git submodule foreach.

>     We have just been dealing with a
>     few of these issues and currently call git directly to sync, init, and
>     then use submodule foreach to reset and clean all submodules before
>     updating.

That is the expected way to deal with it.  The local dashboard
script knows if it needs to preserve the user config or not.

We could also consider adding options for ctest_update to
tell it to init, sync, etc., but it should not be the default.

-Brad
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Marcus D. Hanwell-2
On Tue, Feb 26, 2013 at 4:50 PM, Brad King <[hidden email]> wrote:
> On 2/26/2013 2:52 PM, Jean-Christophe Fillion-Robin wrote:
>> +1 to add these into CTest :) What would be the argument against it ?
>
> Not every project wants every submodule checked out all the time.
> A major use case for them is to have an umbrella project with many
> submodules and the developer may only checkout and work on some.
> Some may even be proprietary and inaccessible to some machines.

Agreed, there are different uses for submodules but it seems like we
are also neglecting a common use case.

>
>>     > However, it seems to me that ctest already does a
>>     > git submodule update --recurse
>
> This is the proper command to update everything that is already
> to configured to checkout in the source tree.
>
>>     > but its missing the --init flag to deal with changes to the
>>     > .gitmodules file.
>>     It also misses git submodule sync to deal with changes in git
>>     submodule URL,
>
> These are all intentionally missing for the above reason.  We
> should honor the user's configuration.  Maybe they intentionally
> use a custom url for a submodule for the branch they test.  We
> should not blow away their configuration by default.
>
>> and reset --hard etc.
>
> We do a reset --hard at the top level but I do not think we
> do it in the submodules.  That may be worth adding, perhaps
> with git submodule foreach.

This would be very helpful, and is a big inconsistency in the current behavior.

>
>>     We have just been dealing with a
>>     few of these issues and currently call git directly to sync, init, and
>>     then use submodule foreach to reset and clean all submodules before
>>     updating.
>
> That is the expected way to deal with it.  The local dashboard
> script knows if it needs to preserve the user config or not.
>
> We could also consider adding options for ctest_update to
> tell it to init, sync, etc., but it should not be the default.
>
I think adding an option would be totally fine, keep the current
default. It can be difficult getting the order right when scripting
and adding a few options to the update command could make the scripts
quite a bit simpler. If this seems like a reasonable path forward
maybe we could sketch out what those options should look like and we
might be able to put a patch together.

Our most common dashboard use case just wants a pristine clone, using
the latest submodule URLs and ideally should clean out any local
changes. We seem to spend quite a bit of time ensuring this all
happens in the right order, as I think others do.

Marcus
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

norulez
In reply to this post by Brad King
I think for the --tags option it is the same, isn't it?

How can I set such option for the checkout/update command?

Best Regards

Am 26.02.2013 um 22:50 schrieb Brad King <[hidden email]>:

> On 2/26/2013 2:52 PM, Jean-Christophe Fillion-Robin wrote:
>> +1 to add these into CTest :) What would be the argument against it ?
>
> Not every project wants every submodule checked out all the time.
> A major use case for them is to have an umbrella project with many
> submodules and the developer may only checkout and work on some.
> Some may even be proprietary and inaccessible to some machines.
>
>>> However, it seems to me that ctest already does a
>>> git submodule update --recurse
>
> This is the proper command to update everything that is already
> to configured to checkout in the source tree.
>
>>> but its missing the --init flag to deal with changes to the
>>> .gitmodules file.
>>    It also misses git submodule sync to deal with changes in git
>>    submodule URL,
>
> These are all intentionally missing for the above reason.  We
> should honor the user's configuration.  Maybe they intentionally
> use a custom url for a submodule for the branch they test.  We
> should not blow away their configuration by default.
>
>> and reset --hard etc.
>
> We do a reset --hard at the top level but I do not think we
> do it in the submodules.  That may be worth adding, perhaps
> with git submodule foreach.
>
>>    We have just been dealing with a
>>    few of these issues and currently call git directly to sync, init, and
>>    then use submodule foreach to reset and clean all submodules before
>>    updating.
>
> That is the expected way to deal with it.  The local dashboard
> script knows if it needs to preserve the user config or not.
>
> We could also consider adding options for ctest_update to
> tell it to init, sync, etc., but it should not be the default.
>
> -Brad
> --
>
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Follow this link to subscribe/unsubscribe:
> http://www.cmake.org/mailman/listinfo/cmake
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Brad King
On 02/27/2013 12:36 AM, NoRulez wrote:
> I think for the --tags option it is the same, isn't it?
>
> How can I set such option for the checkout/update command?

The --tags option belongs to "git fetch" and extra flags can
be added for that by setting CTEST_GIT_UPDATE_OPTIONS before
calling ctest_update.  Try:

 set(CTEST_GIT_UPDATE_OPTIONS "--tags")
 ...
 ctest_update()

Alternatively one may configure the Git repo with

 git config remote.origin.tagopt "--tags"

so that "git fetch" does it automatically.

-Brad
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

norulez
But if I set the option:
set(CTEST_GIT_UPDATE_OPTIONS "--tags")

Then I get the following error message:
Update command failed: "C:/Program Files/Git/cmd/git.cmd" "fetch" "--tags"

Did I miss something?

Thanks in advance
Best Regards

Am 27.02.2013 um 14:01 schrieb Brad King <[hidden email]>:

> On 02/27/2013 12:36 AM, NoRulez wrote:
>> I think for the --tags option it is the same, isn't it?
>>
>> How can I set such option for the checkout/update command?
>
> The --tags option belongs to "git fetch" and extra flags can
> be added for that by setting CTEST_GIT_UPDATE_OPTIONS before
> calling ctest_update.  Try:
>
> set(CTEST_GIT_UPDATE_OPTIONS "--tags")
> ...
> ctest_update()
>
> Alternatively one may configure the Git repo with
>
> git config remote.origin.tagopt "--tags"
>
> so that "git fetch" does it automatically.
>
> -Brad
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Brad King
On 02/28/2013 02:38 AM, NoRulez wrote:
> But if I set the option:
> set(CTEST_GIT_UPDATE_OPTIONS "--tags")
>
> Then I get the following error message:
> Update command failed: "C:/Program Files/Git/cmd/git.cmd" "fetch" "--tags"

Isn't that the command you want?  Perhaps it really fails.
Try running it by hand from a command line.

-Brad
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Kornel Benko

Am Donnerstag, 28. Februar 2013 um 07:07:52, schrieb Brad King <[hidden email]>

> On 02/28/2013 02:38 AM, NoRulez wrote:

> > But if I set the option:

> > set(CTEST_GIT_UPDATE_OPTIONS "--tags")

> >

> > Then I get the following error message:

> > Update command failed: "C:/Program Files/Git/cmd/git.cmd" "fetch" "--tags"

 

Maybe, you should set also the working directory for this command (source tree!).

 

> Isn't that the command you want? Perhaps it really fails.

> Try running it by hand from a command line.

>

> -Brad

 

Kornel

 


--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

norulez
In reply to this post by Brad King
Yes, it is the command which I want.

On the command line the same command works in the source tree.

The command only fails during the CTest run if I set the option.


Am 28.02.2013 um 13:07 schrieb Brad King <[hidden email]>:

> On 02/28/2013 02:38 AM, NoRulez wrote:
>> But if I set the option:
>> set(CTEST_GIT_UPDATE_OPTIONS "--tags")
>>
>> Then I get the following error message:
>> Update command failed: "C:/Program Files/Git/cmd/git.cmd" "fetch" "--tags"
>
> Isn't that the command you want?  Perhaps it really fails.
> Try running it by hand from a command line.
>
> -Brad
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: ctest & git submodules

Brad King
On 02/28/2013 10:34 AM, NoRulez wrote:
> Yes, it is the command which I want.
>
> On the command line the same command works in the source tree.
>
> The command only fails during the CTest run if I set the option.

Read

 Testing/Temporary/LastUpdate*.log

in the test build tree for verbose details.

-Brad
--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake