FindCUDA Separable Compilation and cublas_device

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

FindCUDA Separable Compilation and cublas_device

Jamil Appa
Hi 

   When using FindCUDA with separable compilation combined with device code that uses cublas_device I get missing symbols when linking the intermediate file at line 1758 FindCUDA.cmake

   If I  change line 1758 to include the cublas_device library the symbols are resolved correctly.

   COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} -o ${output_file} -lcublas_device

   Is there a way to pass through a library to the CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS function that I am missing?

   Thanks

 Jamil

Jamil Appa | Co-Founder and Director | Zenotech
Papercut
Tel:+44 (0)7747 606 788Zenotech LTD - Simulation Unlimited
Email:[hidden email]
Web:www.zenotech.com
Papercut
linkedin:Twitter:Location:

Company Registration No : 07926926 | VAT No : 128198591

Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF, UK

Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol BS16 7FR



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindCUDA Separable Compilation and cublas_device

Robert Maynard
Hi,

Have you tried using the 'CUDA_ADD_CUBLAS_TO_TARGET' command on the target that has FindCUDA separable compilation enabled?

On Wed, Nov 1, 2017 at 11:16 AM, Jamil Appa <[hidden email]> wrote:
Hi 

   When using FindCUDA with separable compilation combined with device code that uses cublas_device I get missing symbols when linking the intermediate file at line 1758 FindCUDA.cmake

   If I  change line 1758 to include the cublas_device library the symbols are resolved correctly.

   COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} -o ${output_file} -lcublas_device

   Is there a way to pass through a library to the CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS function that I am missing?

   Thanks

 Jamil

Jamil Appa | Co-Founder and Director | Zenotech
Papercut
Tel:<a href="tel:+44%207747%20606788" value="+447747606788" target="_blank">+44 (0)7747 606 788Zenotech LTD - Simulation Unlimited
Email:[hidden email]
Web:www.zenotech.com
Papercut
linkedin:Twitter:Location:

Company Registration No : 07926926 | VAT No : 128198591

Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF, UK

Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol BS16 7FR



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindCUDA Separable Compilation and cublas_device

Jamil Appa
Hi Robert

  I am using CUDA_ADD_CUBLAS_TO_TARGET but the error happens during the prelink phase when using separable compilation. (nvcc -dlink phase) 

   My guess is that Nvidia have not included cublas_device.a in their dlink phase as you don't have to provide any other libraries. I have asked the Nvidia team if that is the case, but no reply yet.

   The general fix seems to be to modify lines 1759/1772 of FindCUDA.cmake (in gitlab master) to include CUBLAS_LIBRARIES

 COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} ${CUDA_CUBLAS_LIBRARIES} -o ${output_file}

 This will have no impact if the cublas symbols are not used in the object files.

  If you are happy with this I can submit the change for review.

 Jamil

On Thu, 2 Nov 2017 at 13:53 Robert Maynard <[hidden email]> wrote:
Hi,

Have you tried using the 'CUDA_ADD_CUBLAS_TO_TARGET' command on the target that has FindCUDA separable compilation enabled?

On Wed, Nov 1, 2017 at 11:16 AM, Jamil Appa <[hidden email]> wrote:
Hi 

   When using FindCUDA with separable compilation combined with device code that uses cublas_device I get missing symbols when linking the intermediate file at line 1758 FindCUDA.cmake

   If I  change line 1758 to include the cublas_device library the symbols are resolved correctly.

   COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} -o ${output_file} -lcublas_device

   Is there a way to pass through a library to the CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS function that I am missing?

   Thanks

 Jamil

Jamil Appa | Co-Founder and Director | Zenotech
Papercut
Tel:<a href="tel:+44%207747%20606788" value="+447747606788" target="_blank">+44 (0)7747 606 788Zenotech LTD - Simulation Unlimited
Email:[hidden email]
Web:www.zenotech.com
Papercut
linkedin:Twitter:Location:

Company Registration No : 07926926 | VAT No : 128198591

Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF, UK

Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol BS16 7FR



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindCUDA Separable Compilation and cublas_device

Robert Maynard
HI Jamil,

If you are interested in submitting a fix I can review, but I would like to also make sure that this issue doesn't exist inside the new cmake support for CUDA as a first class language. Can you produce a minimal test case that we can rework to use the new support and add to CMake's test suite.

You will also need to track down what version of CUDA first added cublas_device so that we properly guard the inclusion.

On Fri, Nov 3, 2017 at 7:39 AM, Jamil Appa <[hidden email]> wrote:
Hi Robert

  I am using CUDA_ADD_CUBLAS_TO_TARGET but the error happens during the prelink phase when using separable compilation. (nvcc -dlink phase) 

   My guess is that Nvidia have not included cublas_device.a in their dlink phase as you don't have to provide any other libraries. I have asked the Nvidia team if that is the case, but no reply yet.

   The general fix seems to be to modify lines 1759/1772 of FindCUDA.cmake (in gitlab master) to include CUBLAS_LIBRARIES

 COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} ${CUDA_CUBLAS_LIBRARIES} -o ${output_file}

 This will have no impact if the cublas symbols are not used in the object files.

  If you are happy with this I can submit the change for review.

 Jamil


On Thu, 2 Nov 2017 at 13:53 Robert Maynard <[hidden email]> wrote:
Hi,

Have you tried using the 'CUDA_ADD_CUBLAS_TO_TARGET' command on the target that has FindCUDA separable compilation enabled?

On Wed, Nov 1, 2017 at 11:16 AM, Jamil Appa <[hidden email]> wrote:
Hi 

   When using FindCUDA with separable compilation combined with device code that uses cublas_device I get missing symbols when linking the intermediate file at line 1758 FindCUDA.cmake

   If I  change line 1758 to include the cublas_device library the symbols are resolved correctly.

   COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} -o ${output_file} -lcublas_device

   Is there a way to pass through a library to the CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS function that I am missing?

   Thanks

 Jamil

Jamil Appa | Co-Founder and Director | Zenotech
Papercut
Tel:<a href="tel:+44%207747%20606788" value="+447747606788" target="_blank">+44 (0)7747 606 788Zenotech LTD - Simulation Unlimited
Email:[hidden email]
Web:www.zenotech.com
Papercut
linkedin:Twitter:Location:

Company Registration No : 07926926 | VAT No : 128198591

Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF, UK

Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol BS16 7FR



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindCUDA Separable Compilation and cublas_device

Jamil Appa
Hi Robert

   I will put a minimal case together using one of the CUDA samples and sent it through for testing with the new cmake support.

   On your second point, CUDA_CUBLAS_LIBRARIES should be empty if no cublas libraries are present so that use case should be automatically handled unless I am missing something.

   I will create a fork and topic branch with the change and submit it for review.

 Thanks

  Jamil

On Fri, 3 Nov 2017 at 18:08 Robert Maynard <[hidden email]> wrote:
HI Jamil,

If you are interested in submitting a fix I can review, but I would like to also make sure that this issue doesn't exist inside the new cmake support for CUDA as a first class language. Can you produce a minimal test case that we can rework to use the new support and add to CMake's test suite.

You will also need to track down what version of CUDA first added cublas_device so that we properly guard the inclusion.

On Fri, Nov 3, 2017 at 7:39 AM, Jamil Appa <[hidden email]> wrote:
Hi Robert

  I am using CUDA_ADD_CUBLAS_TO_TARGET but the error happens during the prelink phase when using separable compilation. (nvcc -dlink phase) 

   My guess is that Nvidia have not included cublas_device.a in their dlink phase as you don't have to provide any other libraries. I have asked the Nvidia team if that is the case, but no reply yet.

   The general fix seems to be to modify lines 1759/1772 of FindCUDA.cmake (in gitlab master) to include CUBLAS_LIBRARIES

 COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} ${CUDA_CUBLAS_LIBRARIES} -o ${output_file}

 This will have no impact if the cublas symbols are not used in the object files.

  If you are happy with this I can submit the change for review.

 Jamil


On Thu, 2 Nov 2017 at 13:53 Robert Maynard <[hidden email]> wrote:
Hi,

Have you tried using the 'CUDA_ADD_CUBLAS_TO_TARGET' command on the target that has FindCUDA separable compilation enabled?

On Wed, Nov 1, 2017 at 11:16 AM, Jamil Appa <[hidden email]> wrote:
Hi 

   When using FindCUDA with separable compilation combined with device code that uses cublas_device I get missing symbols when linking the intermediate file at line 1758 FindCUDA.cmake

   If I  change line 1758 to include the cublas_device library the symbols are resolved correctly.

   COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} -o ${output_file} -lcublas_device

   Is there a way to pass through a library to the CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS function that I am missing?

   Thanks

 Jamil

Jamil Appa | Co-Founder and Director | Zenotech
Papercut
Tel:<a href="tel:+44%207747%20606788" value="+447747606788" target="_blank">+44 (0)7747 606 788Zenotech LTD - Simulation Unlimited
Email:[hidden email]
Web:www.zenotech.com
Papercut
linkedin:Twitter:Location:

Company Registration No : 07926926 | VAT No : 128198591

Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF, UK

Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol BS16 7FR



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: FindCUDA Separable Compilation and cublas_device

Jamil Appa

Hi Robert

  Find attached a test case that reproduces the error.

 Jamil

On Fri, 3 Nov 2017 at 21:00 Jamil Appa <[hidden email]> wrote:
Hi Robert

   I will put a minimal case together using one of the CUDA samples and sent it through for testing with the new cmake support.

   On your second point, CUDA_CUBLAS_LIBRARIES should be empty if no cublas libraries are present so that use case should be automatically handled unless I am missing something.

   I will create a fork and topic branch with the change and submit it for review.

 Thanks

  Jamil


On Fri, 3 Nov 2017 at 18:08 Robert Maynard <[hidden email]> wrote:
HI Jamil,

If you are interested in submitting a fix I can review, but I would like to also make sure that this issue doesn't exist inside the new cmake support for CUDA as a first class language. Can you produce a minimal test case that we can rework to use the new support and add to CMake's test suite.

You will also need to track down what version of CUDA first added cublas_device so that we properly guard the inclusion.

On Fri, Nov 3, 2017 at 7:39 AM, Jamil Appa <[hidden email]> wrote:
Hi Robert

  I am using CUDA_ADD_CUBLAS_TO_TARGET but the error happens during the prelink phase when using separable compilation. (nvcc -dlink phase) 

   My guess is that Nvidia have not included cublas_device.a in their dlink phase as you don't have to provide any other libraries. I have asked the Nvidia team if that is the case, but no reply yet.

   The general fix seems to be to modify lines 1759/1772 of FindCUDA.cmake (in gitlab master) to include CUBLAS_LIBRARIES

 COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} ${CUDA_CUBLAS_LIBRARIES} -o ${output_file}

 This will have no impact if the cublas symbols are not used in the object files.

  If you are happy with this I can submit the change for review.

 Jamil


On Thu, 2 Nov 2017 at 13:53 Robert Maynard <[hidden email]> wrote:
Hi,

Have you tried using the 'CUDA_ADD_CUBLAS_TO_TARGET' command on the target that has FindCUDA separable compilation enabled?

On Wed, Nov 1, 2017 at 11:16 AM, Jamil Appa <[hidden email]> wrote:
Hi 

   When using FindCUDA with separable compilation combined with device code that uses cublas_device I get missing symbols when linking the intermediate file at line 1758 FindCUDA.cmake

   If I  change line 1758 to include the cublas_device library the symbols are resolved correctly.

   COMMAND ${CUDA_NVCC_EXECUTABLE} ${nvcc_flags} -dlink ${object_files} -o ${output_file} -lcublas_device

   Is there a way to pass through a library to the CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS function that I am missing?

   Thanks

 Jamil

Jamil Appa | Co-Founder and Director | Zenotech
Papercut
Tel:<a href="tel:+44%207747%20606788" value="+447747606788" target="_blank">+44 (0)7747 606 788Zenotech LTD - Simulation Unlimited
Email:[hidden email]
Web:www.zenotech.com
Papercut
linkedin:Twitter:Location:

Company Registration No : 07926926 | VAT No : 128198591

Registered Office : 1 Larkfield Grove, Chepstow, Monmouthshire, NP16 5UF, UK

Address : Bristol & Bath Science Park, Dirac Cres, Emersons Green, Bristol BS16 7FR



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake



--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

cuda-test.tar (13K) Download Attachment