Quantcast

CTest question

classic Classic list List threaded Threaded
5 messages Options
TS
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

CTest question

TS

Hi,

 

I wonder what is the reason that sometimes ctest ends with numerous ‘Failed’ and ‘Timeout’, which ‘ctest –rerun-failed’ reports ‘Passed’. At times the number of these false failures is about one third. I guess that the problem is in the way I set my tests, but I have no clue where to look, or how to ‘debug’ the ctest run.

Any advice is highly appreciated.

 

Thanks,

Ted


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CTest question

Eric Noulard
We lack information on how your test are failing. If they all timeout then may be you should setup a proper timeout for the test. The actual execution speed of a test may depend on the load of your machine or the network (depending on what the test are actually doing).
This may be even more sensible if your test are run in parallel. Do you run ctest with --parallel option? 

Do your tests depends on each other? If so did you setup test dependencies?

Le 17 mai 2017 10:59 PM, "T.Sariyski" <[hidden email]> a écrit :

Hi,

 

I wonder what is the reason that sometimes ctest ends with numerous ‘Failed’ and ‘Timeout’, which ‘ctest –rerun-failed’ reports ‘Passed’. At times the number of these false failures is about one third. I guess that the problem is in the way I set my tests, but I have no clue where to look, or how to ‘debug’ the ctest run.

Any advice is highly appreciated.

 

Thanks,

Ted


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
TS
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CTest question

TS

Hi Eric,

 

I am sorry I did not state the question clear enough. Let me first say that tests does not depend on each other, and  running with or without –parallel does not make much of difference.

 

Here is how I set the tests . This is a Fortran code, it does not return. ADD_TEST command is a call to CMAKE, which (i) executes the Fortran code, and (ii) executes a python script, a metrics of how close the solution is to the reference data. I have a PASS_REGULAR_EXPRESSION property,  passRegex, associated with the test. When a test fails, the output is that the required regular expression ${passRegex} is not found. A test and the python script are attached. Please let me know if more information is needed.

 

Thanks in advance,

Ted

 

 

CMAKE_MINIMUM_REQUIRED(VERSION 2.8)

project(project_name Fortran )

enable_testing()

 

set (SRUN_COMMAND /bin/sh -c "cd ${ CMAKE_CURRENT_BINARY_DIR } &&  $<TARGET_FILE:my_exec> ${PROJECT_NAME}")

    add_custom_target(${PROJECT_NAME}_srun COMMAND ${SRUN_COMMAND})

    set(TARGET_NAME ${PROJECT_NAME}_serial)

    add_test(NAME ${TARGET_NAME}

       WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}

       COMMAND ${CMAKE_COMMAND}

         -D "PRE_DELETE_OUTPUT_FILES:BOOL=true"

         -D "OUTPUT_FILES:STRING=${files_to_delete}"

         -D "PROGRAM1:STRING=${SRUN_COMMAND}"

         -D "PROGRAM2:STRING=python;${assert_case};${hs_work_file};${hs_gold_file};0"        

         -P "run_and_assert.cmake")

 

    set(passRegex "CALCULATION TERMINATED BECAUSE STOP TIME ACHIEVED" "ctest: SUCCESS")

    set_property(TEST ${TARGET_NAME}

                 PROPERTY PASS_REGULAR_EXPRESSION "${passRegex}")

   

 

/> cat run_and_assert.cmake

EXECUTE_PROCESS(COMMAND ${PROGRAM1}

                RESULT_VARIABLE error

                ERROR_FILE err.log

                OUTPUT_FILE out.log

                )

IF(error)

    MESSAGE(FATAL_ERROR "Executing \"${PROGRAM1}\" FAIED! error=${error}")

ENDIF()

 

EXECUTE_PROCESS(COMMAND ${PROGRAM2}

                RESULT_VARIABLE status)

IF(status)

    MESSAGE(SEND_ERROR "Executing \"${PROGRAM2}\" FAILED! status=${status}")

ENDIF()

 

 

 

From: Eric Noulard [mailto:[hidden email]]
Sent: Wednesday, May 17, 2017 10:54 PM
To: T.Sariyski <[hidden email]>
Cc: CMake ML <[hidden email]>
Subject: Re: [CMake] CTest question

 

We lack information on how your test are failing. If they all timeout then may be you should setup a proper timeout for the test. The actual execution speed of a test may depend on the load of your machine or the network (depending on what the test are actually doing).

This may be even more sensible if your test are run in parallel. Do you run ctest with --parallel option? 

 

Do your tests depends on each other? If so did you setup test dependencies?

 

Le 17 mai 2017 10:59 PM, "T.Sariyski" <[hidden email]> a écrit :

Hi,

 

I wonder what is the reason that sometimes ctest ends with numerous ‘Failed’ and ‘Timeout’, which ‘ctest –rerun-failed’ reports ‘Passed’. At times the number of these false failures is about one third. I guess that the problem is in the way I set my tests, but I have no clue where to look, or how to ‘debug’ the ctest run.

Any advice is highly appreciated.

 

Thanks,

Ted


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
TS
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

FW: CTest question

TS
In reply to this post by Eric Noulard

Hi Eric,

 

I noticed something that may relates to the problem. Like I said in my previous mail, add_test is a call to cmake:

 

    ADD_TEST(NAME ${TARGET_NAME}

       …

       COMMAND ${CMAKE_COMMAND}

-DPROGRAM1

-DPROGRAM2 

 

Is there a chance that PROGRAM1 and PROGRAM2 are executed asynchronically? Here is what I noticed. If a test is executed from its own binary directory, ‘ctest -VV’, it always pass. If the same test is executed from the main binary directory, ‘ctest -VV -R zamod’,  it fails because PROGRAM2 is executed before PROGRAM1 is finished. Attached are the screen outputs for both runs.

 

Thanks,

Ted

 

 

================================================================

cd CTEST/zamod

s1002807:zamod tesari$ ctest -VV

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/CTEST/zamod/DartConfiguration.tcl

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/CTEST/zamod/DartConfiguration.tcl

Test project /Users/tesari/build/gnu.dbg/CTEST/zamod

Constructing a list of tests

Done constructing a list of tests

Updating test list for fixtures

Added 0 tests to meet fixture requirements

Checking test dependency graph...

Checking test dependency graph end

test 1

    Start 1: zamod_serial

 

1: Test command: /Applications/CMake.app/Contents/bin/cmake "-D" "PRE_DELETE_OUTPUT_FILES:BOOL=true" "-D" "OUTPUT_FILES:STRING=*kdk*;*.exo;*.jpg;*.png;*.log;*~" "-D" "PROGRAM1:STRING=/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod" "-D" "PROGRAM2:STRING=python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0" "-P" "run_and_assert.cmake"

1: Test timeout computed to be: 9.99988e+06

1: -- Executing "/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod"

1: -- Executing "python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0"

1: /Users/tesari/kodiak/trunk/benchmark/Scripts

1: Number of arguments: 4 arguments.

1: Argument List: ['/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py', '/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk', '/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk', '0']

1: /Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py called with  3  arguments:

1: 1 /Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk

1: 2 /Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk

1: 3 0

1:            zamod             Q0.4 PASS

1:            zamod             Q0.5 PASS

1: Kodiak ctest: SUCCESS

1/1 Test #1: zamod_serial .....................   Passed    5.73 sec

 

 

================================================================

s1002807:gnu.dbg tesari$ ctest -VV -R zamod

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

Parse Config file:/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

Parse Config file:/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

Test project /Users/tesari/build/gnu.dbg

Constructing a list of tests

Done constructing a list of tests

Updating test list for fixtures

Added 0 tests to meet fixture requirements

Checking test dependency graph...

Checking test dependency graph end

test 113

    Start 113: zamod_serial

 

113: Test command: /Applications/CMake.app/Contents/bin/cmake "-D" "PRE_DELETE_OUTPUT_FILES:BOOL=true" "-D" "OUTPUT_FILES:STRING=*kdk*;*.exo;*.jpg;*.png;*.log;*~" "-D" "PROGRAM1:STRING=/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod" "-D" "PROGRAM2:STRING=python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0" "-P" "run_and_assert.cmake"

113: Test timeout computed to be: 1500

113: -- Executing "/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod"

113: -- Executing "python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0"

113: /Users/tesari/kodiak/trunk/benchmark/Scripts

113: Number of arguments: 4 arguments.

113: Argument List: ['/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py', '/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk', '/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk', '0']

113: /Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py called with  3  arguments:

113: 1 /Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk

113: 2 /Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk

113: 3 0

113: /Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk  does not exist!

 

 

 

From: Eric Noulard [[hidden email]]
Sent: Wednesday, May 17, 2017 10:54 PM
To: T.Sariyski <[hidden email]>
Cc: CMake ML <[hidden email]>
Subject: Re: [CMake] CTest question

 

We lack information on how your test are failing. If they all timeout then may be you should setup a proper timeout for the test. The actual execution speed of a test may depend on the load of your machine or the network (depending on what the test are actually doing).

This may be even more sensible if your test are run in parallel. Do you run ctest with --parallel option? 

 

Do your tests depends on each other? If so did you setup test dependencies?

 

Le 17 mai 2017 10:59 PM, "T.Sariyski" <[hidden email]> a écrit :

Hi,

 

I wonder what is the reason that sometimes ctest ends with numerous ‘Failed’ and ‘Timeout’, which ‘ctest –rerun-failed’ reports ‘Passed’. At times the number of these false failures is about one third. I guess that the problem is in the way I set my tests, but I have no clue where to look, or how to ‘debug’ the ctest run.

Any advice is highly appreciated.

 

Thanks,

Ted


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FW: CTest question

Eric Noulard
Hi,

I don't that 2 execute_process in sequence in a cmake script could be executed out-of-order but I let CMake developer answer more firmly about that.
The order of each test created by add_test may vary though.

2017-05-18 19:32 GMT+02:00 T.Sariyski <[hidden email]>:

Hi Eric,

 

I noticed something that may relates to the problem. Like I said in my previous mail, add_test is a call to cmake:

 

    ADD_TEST(NAME ${TARGET_NAME}

       …

       COMMAND ${CMAKE_COMMAND}

-DPROGRAM1

-DPROGRAM2 

 

Is there a chance that PROGRAM1 and PROGRAM2 are executed asynchronically? Here is what I noticed. If a test is executed from its own binary directory, ‘ctest -VV’, it always pass. If the same test is executed from the main binary directory, ‘ctest -VV -R zamod’,  it fails because PROGRAM2 is executed before PROGRAM1 is finished. Attached are the screen outputs for both runs.

 

Thanks,

Ted

 

 

================================================================

cd CTEST/zamod

s1002807:zamod tesari$ ctest -VV

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/CTEST/zamod/DartConfiguration.tcl

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/CTEST/zamod/DartConfiguration.tcl

Test project /Users/tesari/build/gnu.dbg/CTEST/zamod

Constructing a list of tests

Done constructing a list of tests

Updating test list for fixtures

Added 0 tests to meet fixture requirements

Checking test dependency graph...

Checking test dependency graph end

test 1

    Start 1: zamod_serial

 

1: Test command: /Applications/CMake.app/Contents/bin/cmake "-D" "PRE_DELETE_OUTPUT_FILES:BOOL=true" "-D" "OUTPUT_FILES:STRING=*kdk*;*.exo;*.jpg;*.png;*.log;*~" "-D" "PROGRAM1:STRING=/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod" "-D" "PROGRAM2:STRING=python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0" "-P" "run_and_assert.cmake"

1: Test timeout computed to be: 9.99988e+06

1: -- Executing "/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod"

1: -- Executing "python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0"

1: /Users/tesari/kodiak/trunk/benchmark/Scripts

1: Number of arguments: 4 arguments.

1: Argument List: ['/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py', '/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk', '/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk', '0']

1: /Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py called with  3  arguments:

1: 1 /Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk

1: 2 /Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk

1: 3 0

1:            zamod             Q0.4 PASS

1:            zamod             Q0.5 PASS

1: Kodiak ctest: SUCCESS

1/1 Test #1: zamod_serial .....................   Passed    5.73 sec

 

 

================================================================

s1002807:gnu.dbg tesari$ ctest -VV -R zamod

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

Parse Config file:/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

UpdateCTestConfiguration  from :/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

Parse Config file:/Users/tesari/build/gnu.dbg/DartConfiguration.tcl

Test project /Users/tesari/build/gnu.dbg

Constructing a list of tests

Done constructing a list of tests

Updating test list for fixtures

Added 0 tests to meet fixture requirements

Checking test dependency graph...

Checking test dependency graph end

test 113

    Start 113: zamod_serial

 

113: Test command: /Applications/CMake.app/Contents/bin/cmake "-D" "PRE_DELETE_OUTPUT_FILES:BOOL=true" "-D" "OUTPUT_FILES:STRING=*kdk*;*.exo;*.jpg;*.png;*.log;*~" "-D" "PROGRAM1:STRING=/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod" "-D" "PROGRAM2:STRING=python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0" "-P" "run_and_assert.cmake"

113: Test timeout computed to be: 1500

113: -- Executing "/bin/sh;-c;cd /Users/tesari/build/gnu.dbg/CTEST/zamod &&  /Users/tesari/build/gnu.dbg/bin/kodiak zamod"

113: -- Executing "python;/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py;/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk;/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk;0"

113: /Users/tesari/kodiak/trunk/benchmark/Scripts

113: Number of arguments: 4 arguments.

113: Argument List: ['/Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py', '/Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk', '/Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk', '0']

113: /Users/tesari/kodiak/trunk/benchmark/Scripts/assert_case.py called with  3  arguments:

113: 1 /Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk

113: 2 /Users/tesari/kodiak/trunk/CTEST/zamod/../GOLD/zamod/hskdk

113: 3 0

113: /Users/tesari/build/gnu.dbg/CTEST/zamod/hskdk  does not exist!

 

 

 

From: Eric Noulard [[hidden email]]
Sent: Wednesday, May 17, 2017 10:54 PM
To: T.Sariyski <[hidden email]>
Cc: CMake ML <[hidden email]>
Subject: Re: [CMake] CTest question

 

We lack information on how your test are failing. If they all timeout then may be you should setup a proper timeout for the test. The actual execution speed of a test may depend on the load of your machine or the network (depending on what the test are actually doing).

This may be even more sensible if your test are run in parallel. Do you run ctest with --parallel option? 

 

Do your tests depends on each other? If so did you setup test dependencies?

 

Le 17 mai 2017 10:59 PM, "T.Sariyski" <[hidden email]> a écrit :

Hi,

 

I wonder what is the reason that sometimes ctest ends with numerous ‘Failed’ and ‘Timeout’, which ‘ctest –rerun-failed’ reports ‘Passed’. At times the number of these false failures is about one third. I guess that the problem is in the way I set my tests, but I have no clue where to look, or how to ‘debug’ the ctest run.

Any advice is highly appreciated.

 

Thanks,

Ted


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--
Eric

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Loading...