Submitting build and test results to dashboard separately

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Submitting build and test results to dashboard separately

Moreland, Kenneth
Submitting build and test results to dashboard separately I am setting up a ParaView dashboard for a cluster in which I have to compile on one computer and then submit a job to run parallel tests on another computer. This means I have to run one dashboard script to update and build the code.  At the end of this script I submit a parallel job to run the tests.  The problem I am having is that I cannot seem to submit the results to the dashboard in such a way that I get both the build and test information on the same entry.

My current configuration is as follows.  To update, configure, and build the latest version I have a CTest script like the following:

-Setup CMake variables-

# Clear out binary directory and set defaults.
CTEST_EMPTY_BINARY_DIRECTORY(${CTEST_BINARY_DIRECTORY})
FILE(WRITE "${CTEST_BINARY_DIRECTORY}/CMakeCache.txt" "${CTEST_INITIAL_CACHE}")
FILE(MAKE_DIRECTORY "${CTEST_BINARY_DIRECTORY}/Testing/Notes")

CTEST_START(${CTEST_BUILD_MODE})
FOREACH(update_dir ${CTEST_EXTRA_UPDATE_DIRECTORIES} ${CTEST_SOURCE_DIRECTORY})
  CTEST_UPDATE(SOURCE ${update_dir})
ENDFOREACH(update_dir)
CTEST_CONFIGURE(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_READ_CUSTOM_FILES("${CTEST_BINARY_DIRECTORY}") # Reads CTestCustom.ctest files
CTEST_BUILD(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_SUBMIT()

-
Submit parallel test job-

When the parallel test job runs, it executes this script.

-Setup CMake variables-

CTEST_START(${CTEST_BUILD_MODE})
CTEST_TEST(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_SUBMIT()

I am sure that the CMake variables are setup the same in both the build and test scripts; I am actually using the same script to setup both.

Now here is the problem: each of the
CTEST_SUBMIT()  create their own entry in the dashboard.  The first one creates one that reports on the build but is missing the test information.  The second one has the results of the test but has bogus information for the build.  Does anyone know how to get the the second submit to simply add the test information to the existing build information?  I tried simply removing the CTEST_START command from the test script, but in that case the submission fails outright.  I get output like the following:

Test project /home/kmorel/dashboard/builds/ParaView3Experimental
Cannot create directory /Testing/Temporary
Cannot create log file: LastTest.log
    Start 1: QVTK-TestQtInitialization
1/1 Test #1: QVTK-TestQtInitialization ........   Passed    9.38 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =  11.53 sec
Cannot create directory /Testing
Cannot open notes file
   Use HTTP Proxy: wwwproxy.sandia.gov:80
Cannot create directory /Testing/Temporary
Cannot create log file: LastSubmit.log
Submit files (using http)
   Using HTTP submit method
   Drop site:<a href="http://">http://
   Submission successful
Error in read script: /home/kmorel/dashboard/scripts/ParaView3Experimental.cmake

Any ideas?

-Ken

   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********  
*** *** ***  email: kmorel@...
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Submitting build and test results to dashboard separately

Alexander Neundorf-3
On Wednesday 16 December 2009, Moreland, Kenneth wrote:

> I am setting up a ParaView dashboard for a cluster in which I have to
> compile on one computer and then submit a job to run parallel tests on
> another computer. This means I have to run one dashboard script to update
> and build the code.  At the end of this script I submit a parallel job to
> run the tests.  The problem I am having is that I cannot seem to submit the
> results to the dashboard in such a way that I get both the build and test
> information on the same entry.
>
> My current configuration is as follows.  To update, configure, and build
> the latest version I have a CTest script like the following:
>
> -Setup CMake variables-
...

> I am sure that the CMake variables are setup the same in both the build and
> test scripts; I am actually using the same script to setup both.
>
> Now here is the problem: each of the CTEST_SUBMIT()  create their own entry
> in the dashboard.  The first one creates one that reports on the build but
> is missing the test information.  The second one has the results of the
> test but has bogus information for the build.  Does anyone know how to get
> the the second submit to simply add the test information to the existing
> build information?  I tried simply removing the CTEST_START command from
> the test script, but in that case the submission fails outright.  I get
> output like the following:
>
> Test project /home/kmorel/dashboard/builds/ParaView3Experimental
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastTest.log
>     Start 1: QVTK-TestQtInitialization
> 1/1 Test #1: QVTK-TestQtInitialization ........   Passed    9.38 sec
>
> 100% tests passed, 0 tests failed out of 1
>
> Total Test time (real) =  11.53 sec
> Cannot create directory /Testing
> Cannot open notes file
>    Use HTTP Proxy: wwwproxy.sandia.gov:80
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastSubmit.log
> Submit files (using http)
>    Using HTTP submit method
>    Drop site:http://
>    Submission successful
> Error in read script:
> /home/kmorel/dashboard/scripts/ParaView3Experimental.cmake
>
> Any ideas?
>

Does CTEST_SUBMIT(PARTS ... ) what you want ?
It's here under "CDash Subprojects" :
http://www.kitware.com/products/archive/kitware_quarterly0709.pdf

Alex
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Submitting build and test results to dashboard separately

Moreland, Kenneth
Re: [CMake] Submitting build and test results to dashboard separately No.  That does not help.  I still get two entries on the dashboard.  Is there anything else I can do?

-Ken


On 12/16/09 2:38 PM, "Alexander Neundorf" <a.neundorf-work@...> wrote:

On Wednesday 16 December 2009, Moreland, Kenneth wrote:
> I am setting up a ParaView dashboard for a cluster in which I have to
> compile on one computer and then submit a job to run parallel tests on
> another computer. This means I have to run one dashboard script to update
> and build the code.  At the end of this script I submit a parallel job to
> run the tests.  The problem I am having is that I cannot seem to submit the
> results to the dashboard in such a way that I get both the build and test
> information on the same entry.
>
> My current configuration is as follows.  To update, configure, and build
> the latest version I have a CTest script like the following:
>
> -Setup CMake variables-
...
> I am sure that the CMake variables are setup the same in both the build and
> test scripts; I am actually using the same script to setup both.
>
> Now here is the problem: each of the CTEST_SUBMIT()  create their own entry
> in the dashboard.  The first one creates one that reports on the build but
> is missing the test information.  The second one has the results of the
> test but has bogus information for the build.  Does anyone know how to get
> the the second submit to simply add the test information to the existing
> build information?  I tried simply removing the CTEST_START command from
> the test script, but in that case the submission fails outright.  I get
> output like the following:
>
> Test project /home/kmorel/dashboard/builds/ParaView3Experimental
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastTest.log
>     Start 1: QVTK-TestQtInitialization
> 1/1 Test #1: QVTK-TestQtInitialization ........   Passed    9.38 sec
>
> 100% tests passed, 0 tests failed out of 1
>
> Total Test time (real) =  11.53 sec
> Cannot create directory /Testing
> Cannot open notes file
>    Use HTTP Proxy: wwwproxy.sandia.gov:80
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastSubmit.log
> Submit files (using http)
>    Using HTTP submit method
>    Drop site:<a href="http://">http://
>    Submission successful
> Error in read script:
> /home/kmorel/dashboard/scripts/ParaView3Experimental.cmake
>
> Any ideas?
>

Does CTEST_SUBMIT(PARTS ... ) what you want ?
It's here under "CDash Subprojects" :
http://www.kitware.com/products/archive/kitware_quarterly0709.pdf

Alex
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake




   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********  
*** *** ***  email: kmorel@...
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Submitting build and test results to dashboard separately

David Cole
This sounds like we need to change ctest such that you can do a submit in an "already ctest_start()-ed" tree without getting the error.

Your first instinct, to try leaving off the ctest_start() in the 2nd script sounds completely reasonable to me.

If you are running multiple scripts sequentially, there's no good reason I can think of that the 2nd and later scripts can't reuse the TAG from the first ctest_start call.

Can you submit a bug to the bug tracker requesting this ability...?


Thanks,
David


On Thu, Dec 17, 2009 at 8:37 AM, Moreland, Kenneth <[hidden email]> wrote:
No.  That does not help.  I still get two entries on the dashboard.  Is there anything else I can do?

-Ken



On 12/16/09 2:38 PM, "Alexander Neundorf" <a.neundorf-work@...> wrote:

On Wednesday 16 December 2009, Moreland, Kenneth wrote:
> I am setting up a ParaView dashboard for a cluster in which I have to
> compile on one computer and then submit a job to run parallel tests on
> another computer. This means I have to run one dashboard script to update
> and build the code.  At the end of this script I submit a parallel job to
> run the tests.  The problem I am having is that I cannot seem to submit the
> results to the dashboard in such a way that I get both the build and test
> information on the same entry.
>
> My current configuration is as follows.  To update, configure, and build
> the latest version I have a CTest script like the following:
>
> -Setup CMake variables-
...
> I am sure that the CMake variables are setup the same in both the build and
> test scripts; I am actually using the same script to setup both.
>
> Now here is the problem: each of the CTEST_SUBMIT()  create their own entry
> in the dashboard.  The first one creates one that reports on the build but
> is missing the test information.  The second one has the results of the
> test but has bogus information for the build.  Does anyone know how to get
> the the second submit to simply add the test information to the existing
> build information?  I tried simply removing the CTEST_START command from
> the test script, but in that case the submission fails outright.  I get
> output like the following:
>
> Test project /home/kmorel/dashboard/builds/ParaView3Experimental
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastTest.log
>     Start 1: QVTK-TestQtInitialization
> 1/1 Test #1: QVTK-TestQtInitialization ........   Passed    9.38 sec
>
> 100% tests passed, 0 tests failed out of 1
>
> Total Test time (real) =  11.53 sec
> Cannot create directory /Testing
> Cannot open notes file
>    Use HTTP Proxy: wwwproxy.sandia.gov:80
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastSubmit.log
> Submit files (using http)
>    Using HTTP submit method
>    Drop site:http://
>    Submission successful
> Error in read script:
> /home/kmorel/dashboard/scripts/ParaView3Experimental.cmake
>
> Any ideas?
>

Does CTEST_SUBMIT(PARTS ... ) what you want ?
It's here under "CDash Subprojects" :
http://www.kitware.com/products/archive/kitware_quarterly0709.pdf

Alex
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake




   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********  
*** *** ***  email: kmorel@...
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Submitting build and test results to dashboard separately

Moreland, Kenneth
Re: [CMake] Submitting build and test results to dashboard separately Done.  Thanks.

http://www.paraview.org/Bug/view.php?id=10060

-Ken


On 12/17/09 6:49 AM, "David Cole" <david.cole@...> wrote:

This sounds like we need to change ctest such that you can do a submit in an "already ctest_start()-ed" tree without getting the error.

Your first instinct, to try leaving off the ctest_start() in the 2nd script sounds completely reasonable to me.

If you are running multiple scripts sequentially, there's no good reason I can think of that the 2nd and later scripts can't reuse the TAG from the first ctest_start call.

Can you submit a bug to the bug tracker requesting this ability...?


Thanks,
David


On Thu, Dec 17, 2009 at 8:37 AM, Moreland, Kenneth <kmorel@...> wrote:
No.  That does not help.  I still get two entries on the dashboard.  Is there anything else I can do?

-Ken



On 12/16/09 2:38 PM, "Alexander Neundorf" <a.neundorf-work@... <http://a.neundorf-work@...> > wrote:

On Wednesday 16 December 2009, Moreland, Kenneth wrote:
> I am setting up a ParaView dashboard for a cluster in which I have to
> compile on one computer and then submit a job to run parallel tests on
> another computer. This means I have to run one dashboard script to update
> and build the code.  At the end of this script I submit a parallel job to
> run the tests.  The problem I am having is that I cannot seem to submit the
> results to the dashboard in such a way that I get both the build and test
> information on the same entry.
>
> My current configuration is as follows.  To update, configure, and build
> the latest version I have a CTest script like the following:
>
> -Setup CMake variables-
...
> I am sure that the CMake variables are setup the same in both the build and
> test scripts; I am actually using the same script to setup both.
>
> Now here is the problem: each of the CTEST_SUBMIT()  create their own entry
> in the dashboard.  The first one creates one that reports on the build but
> is missing the test information.  The second one has the results of the
> test but has bogus information for the build.  Does anyone know how to get
> the the second submit to simply add the test information to the existing
> build information?  I tried simply removing the CTEST_START command from
> the test script, but in that case the submission fails outright.  I get
> output like the following:
>
> Test project /home/kmorel/dashboard/builds/ParaView3Experimental
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastTest.log
>     Start 1: QVTK-TestQtInitialization
> 1/1 Test #1: QVTK-TestQtInitialization ........   Passed    9.38 sec
>
> 100% tests passed, 0 tests failed out of 1
>
> Total Test time (real) =  11.53 sec
> Cannot create directory /Testing
> Cannot open notes file
>    Use HTTP Proxy: wwwproxy.sandia.gov:80 <http://wwwproxy.sandia.gov:80>
> Cannot create directory /Testing/Temporary
> Cannot create log file: LastSubmit.log
> Submit files (using http)
>    Using HTTP submit method
>    Drop site:<a href="http://">http://
>    Submission successful
> Error in read script:
> /home/kmorel/dashboard/scripts/ParaView3Experimental.cmake
>
> Any ideas?
>

Does CTEST_SUBMIT(PARTS ... ) what you want ?
It's here under "CDash Subprojects" :
http://www.kitware.com/products/archive/kitware_quarterly0709.pdf

Alex
_______________________________________________
Powered by www.kitware.com <http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake




   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********  
*** *** ***  email: kmorel@... <http://kmorel@...>
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel


_______________________________________________
Powered by www.kitware.com <http://www.kitware.com>

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake




   ****      Kenneth Moreland
    ***      Sandia National Laboratories
***********  
*** *** ***  email: kmorel@...
**  ***  **  phone: (505) 844-8919
    ***      web:   http://www.cs.unm.edu/~kmorel


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake