Executing python though CMake and linking libraries

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Executing python though CMake and linking libraries

Kit Chambers
I have a Cmake custom target which runs a python script:

add_custom_target(run
                 COMMAND python myscript.py
                  )

And the script myscript.py imports a Compiled library

import myproj.mylilb

where mylib is actually a C++ library generated with python bindings (using vtkPython). This intern links against some other C++ libraries I use etc.


When I run the script normally from the command line everything works fine, but if I try to run the target I get linker errors.

$ make run

Traceback (most recent call last):
 File "myscript.py", line 8, in <module>
   import myproj.mylilb
ImportError: dlopen(/Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so, 2): Library not loaded: libSpProcSupport.so
 Referenced from: /Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so
 Reason: image not found

To me it looks like my LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are not being passed through CMake to Python so it cannot find all the necessary sub libraries. However, i cannot work out how to pass this information through.

Any help would be greatly appreciated.

Cheers

Kit
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Executing python though CMake and linking libraries

tonka tonka
Hey,

I think your actual working directory is wrong. I would recommend to use absolute paths for the python interperter (there is a find_package module for the interpreter) and maybe also for your script. The custom command support a workingdirectory variable as far as i know.

Hope that help you.

Greetings
Tonka

> Am 25.11.2016 um 10:04 schrieb Kit Chambers <[hidden email]>:
>
> I have a Cmake custom target which runs a python script:
>
> add_custom_target(run
>                 COMMAND python myscript.py
>                  )
>
> And the script myscript.py imports a Compiled library
>
> import myproj.mylilb
>
> where mylib is actually a C++ library generated with python bindings (using vtkPython). This intern links against some other C++ libraries I use etc.
>
>
> When I run the script normally from the command line everything works fine, but if I try to run the target I get linker errors.
>
> $ make run
>
> Traceback (most recent call last):
> File "myscript.py", line 8, in <module>
>   import myproj.mylilb
> ImportError: dlopen(/Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so, 2): Library not loaded: libSpProcSupport.so
> Referenced from: /Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so
> Reason: image not found
>
> To me it looks like my LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are not being passed through CMake to Python so it cannot find all the necessary sub libraries. However, i cannot work out how to pass this information through.
>
> Any help would be greatly appreciated.
>
> Cheers
>
> Kit
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Executing python though CMake and linking libraries

Kit Chambers

Hmmm… I get the same error if I use absolute paths.

I don’t really understand why the shell environment used to launch CMake and make is not being inherited by the executed commands.

Kit

> On 25 Nov 2016, at 13:23, [hidden email] wrote:
>
> Hey,
>
> I think your actual working directory is wrong. I would recommend to use absolute paths for the python interperter (there is a find_package module for the interpreter) and maybe also for your script. The custom command support a workingdirectory variable as far as i know.
>
> Hope that help you.
>
> Greetings
> Tonka
>
>> Am 25.11.2016 um 10:04 schrieb Kit Chambers <[hidden email]>:
>>
>> I have a Cmake custom target which runs a python script:
>>
>> add_custom_target(run
>>                COMMAND python myscript.py
>>                 )
>>
>> And the script myscript.py imports a Compiled library
>>
>> import myproj.mylilb
>>
>> where mylib is actually a C++ library generated with python bindings (using vtkPython). This intern links against some other C++ libraries I use etc.
>>
>>
>> When I run the script normally from the command line everything works fine, but if I try to run the target I get linker errors.
>>
>> $ make run
>>
>> Traceback (most recent call last):
>> File "myscript.py", line 8, in <module>
>>  import myproj.mylilb
>> ImportError: dlopen(/Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so, 2): Library not loaded: libSpProcSupport.so
>> Referenced from: /Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so
>> Reason: image not found
>>
>> To me it looks like my LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are not being passed through CMake to Python so it cannot find all the necessary sub libraries. However, i cannot work out how to pass this information through.
>>
>> Any help would be greatly appreciated.
>>
>> Cheers
>>
>> Kit
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Executing python though CMake and linking libraries

Kit Chambers
In reply to this post by tonka tonka

I just got this working, so am posting to the list in case this helps someone else in the future. Essentially you invoke the python script through CMakes -E command and specify your library paths so it picks up all your shared libraries. Something like:

add_custom_target(run
               COMMAND cmake "-E" "env"  “LD_LIBRARY_PATH=/your/library/path DYLD_LIBRARY_PATH=/library/path/for/mac/" “python” “./myscript.py”
                WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
                )

However, I will be the first to admit this is not a particularly elegant solution so I would be interested to here if anyone out there has a better way of getting invoked commands to pick up the environment paths correctly.

Kit




> On 25 Nov 2016, at 13:23, [hidden email] wrote:
>
> Hey,
>
> I think your actual working directory is wrong. I would recommend to use absolute paths for the python interperter (there is a find_package module for the interpreter) and maybe also for your script. The custom command support a workingdirectory variable as far as i know.
>
> Hope that help you.
>
> Greetings
> Tonka
>
>> Am 25.11.2016 um 10:04 schrieb Kit Chambers <[hidden email]>:
>>
>> I have a Cmake custom target which runs a python script:
>>
>> add_custom_target(run
>>                COMMAND python myscript.py
>>                 )
>>
>> And the script myscript.py imports a Compiled library
>>
>> import myproj.mylilb
>>
>> where mylib is actually a C++ library generated with python bindings (using vtkPython). This intern links against some other C++ libraries I use etc.
>>
>>
>> When I run the script normally from the command line everything works fine, but if I try to run the target I get linker errors.
>>
>> $ make run
>>
>> Traceback (most recent call last):
>> File "myscript.py", line 8, in <module>
>>  import myproj.mylilb
>> ImportError: dlopen(/Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so, 2): Library not loaded: libSpProcSupport.so
>> Referenced from: /Users/kitchambers/Applications/ogFramework/lib/python2.7/site-packages/myproj/mylib.so
>> Reason: image not found
>>
>> To me it looks like my LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are not being passed through CMake to Python so it cannot find all the necessary sub libraries. However, i cannot work out how to pass this information through.
>>
>> Any help would be greatly appreciated.
>>
>> Cheers
>>
>> Kit
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/cmake

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Executing python though CMake and linking libraries

Matthew Woehlke-3
In reply to this post by Kit Chambers
On 2016-11-25 04:04, Kit Chambers wrote:
> I have a Cmake custom target which runs a python script:
>
> add_custom_target(run
>                  COMMAND python myscript.py
>                   )
>
> ...
>
> To me it looks like my LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are not being passed through CMake to Python so it cannot find all the necessary sub libraries. However, i cannot work out how to pass this information through.

As written, your custom target will be run with whatever environment
your build tool decides to use. *Probably* that will be the environment
when you actually run the build tool (make, ninja, etc.), but it's
possible some build too might monkey with the environment in which it
runs build commands. Also, requiring people building your software to
modify their local environment is probably not ideal.

You probably want to wrap your custom target's command with
`${CMAKE_COMMAND} -E env ...` to ensure it gets the necessary
environment. (Also, you should probably use `${PYTHON_EXECUTABLE}`
instead of `python`, and, similarly, note `${CMAKE_COMMAND}` instead of
`cmake`.)

(Personally, I wouldn't mind seeing CMake learn an `ENVIRONMENT` option
to add_custom_command / add_custom_target that would set up this
wrapping automatically.)

--
Matthew

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Executing python though CMake and linking libraries

Konstantin Tokarev


28.11.2016, 21:53, "Matthew Woehlke" <[hidden email]>:

> On 2016-11-25 04:04, Kit Chambers wrote:
>>  I have a Cmake custom target which runs a python script:
>>
>>  add_custom_target(run
>>                   COMMAND python myscript.py
>>                    )
>>
>>  ...
>>
>>  To me it looks like my LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are not being passed through CMake to Python so it cannot find all the necessary sub libraries. However, i cannot work out how to pass this information through.
>
> As written, your custom target will be run with whatever environment
> your build tool decides to use. *Probably* that will be the environment
> when you actually run the build tool (make, ninja, etc.), but it's
> possible some build too might monkey with the environment in which it
> runs build commands. Also, requiring people building your software to
> modify their local environment is probably not ideal.
>
> You probably want to wrap your custom target's command with
> `${CMAKE_COMMAND} -E env ...` to ensure it gets the necessary
> environment. (Also, you should probably use `${PYTHON_EXECUTABLE}`
> instead of `python`, and, similarly, note `${CMAKE_COMMAND}` instead of
> `cmake`.)
>
> (Personally, I wouldn't mind seeing CMake learn an `ENVIRONMENT` option
> to add_custom_command / add_custom_target that would set up this
> wrapping automatically.)

+1

>
> --
> Matthew
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake

--
Regards,
Konstantin
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake