Installing Python files

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Installing Python files

sandrews
Hi,

My project is primarily C and C++, but also includes some Python code.  I can get everything to build fine, and my CMakeLists.txt files uses the INSTALL(TARGETS ...) to so that the user can install the compiled code to the proper place.  However, part of the installation is to copy the Python files over to their proper places, too, and this isn't working.

First of all, where should Python files go?  On my Mac, I put them in /usr/local/lib/python2.5/site-packages.  Of course though, this directory would be different if I had a different version of Python.  Also, I expect it's different on different platforms.  Does CMake automatically know where the Python files should go and, if so, how do I access that?  (For example, the INSTALL(TARGETS...) command does know where targets are supposed to go for different platforms.)

Next, there are a lot of Python files, so I don't want to just dump the whole collection in my site-packages directory.  Instead, I want to create a subdirectory for them, within my site-packages directory.  How do I do this?  This directory should be created when the user enters "sudo make install", and not during CMake's configuration process.

Thank you!
-Steve

Reply | Threaded
Open this post in threaded view
|

Re: Installing Python files

Andreas Pakulat-2
Hi,

Am Dienstag, 26. März 2013 schrieb sandrews :
Hi,

My project is primarily C and C++, but also includes some Python code.  I
can get everything to build fine, and my CMakeLists.txt files uses the
INSTALL(TARGETS ...) to so that the user can install the compiled code to
the proper place.  However, part of the installation is to copy the Python
files over to their proper places, too, and this isn't working.

First of all, where should Python files go?  On my Mac, I put them in
/usr/local/lib/python2.5/site-packages.  Of course though, this directory
would be different if I had a different version of Python.  Also, I expect
it's different on different platforms.  Does CMake automatically know where
the Python files should go and, if so, how do I access that?  (For example,
the INSTALL(TARGETS...) command does know where targets are supposed to go
for different platforms.)
 
No, cmake does not know about this as python apps are usually installed with python tools like distutils, setuptools etc. That beingsaid cmake does ship modules to find a python interpreter and that one can be queried about its search path. Along with the install command (using the FILES variant) you can have cmake copy your python code where you want it.

However you should think twice before writing cmake code that installs stuff outside thecmake prefix. This is usually unexpected by cmake users, can turn out to be impossible (the user might not be able to writeto thepython install directory) or trip up packaging tools (such as cpack).

On the other hand, having python files in something like $HOME/myapp does require an extra environment variable to be set in order for python to find the code there. Thats something that python users are possibly used to though.

Andreas

--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Installing Python files

sandrews
That's very helpful.  Thank you.

My other question was about how to get CMake to create directories at installation time.  Do you, or someone else, have suggestions about that?  Despite your advice, I think that I want the "make install" step to put Python files into /usr/local/lib/python2.5/site-packages/moleculizer, at least as the default path.  Depending on what the user's system already has, I may need to create one or more levels of this hierarchy.  Do you know if there is a way to create this directory structure, as needed, during the install step?

Thanks,
-Steve


On Tue, Mar 26, 2013 at 4:28 PM, Andreas Pakulat <[hidden email]> wrote:
Hi,

Am Dienstag, 26. März 2013 schrieb sandrews :

Hi,

My project is primarily C and C++, but also includes some Python code.  I
can get everything to build fine, and my CMakeLists.txt files uses the
INSTALL(TARGETS ...) to so that the user can install the compiled code to
the proper place.  However, part of the installation is to copy the Python
files over to their proper places, too, and this isn't working.

First of all, where should Python files go?  On my Mac, I put them in
/usr/local/lib/python2.5/site-packages.  Of course though, this directory
would be different if I had a different version of Python.  Also, I expect
it's different on different platforms.  Does CMake automatically know where
the Python files should go and, if so, how do I access that?  (For example,
the INSTALL(TARGETS...) command does know where targets are supposed to go
for different platforms.)
 
No, cmake does not know about this as python apps are usually installed with python tools like distutils, setuptools etc. That beingsaid cmake does ship modules to find a python interpreter and that one can be queried about its search path. Along with the install command (using the FILES variant) you can have cmake copy your python code where you want it.

However you should think twice before writing cmake code that installs stuff outside thecmake prefix. This is usually unexpected by cmake users, can turn out to be impossible (the user might not be able to writeto thepython install directory) or trip up packaging tools (such as cpack).

On the other hand, having python files in something like $HOME/myapp does require an extra environment variable to be set in order for python to find the code there. Thats something that python users are possibly used to though.

Andreas


--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Installing Python files

Hendrik Sattler
Am 2013-03-27 05:46, schrieb Steve Andrews:

> Thats very helpful.  Thank you.
>
> My other question was about how to get CMake to create directories at
> installation time.  Do you, or someone else, have suggestions about
> that?  Despite your advice, I think that I want the "make install"
> step to put Python files into
> /usr/local/lib/python2.5/site-packages/moleculizer, at least as the
> default path.  Depending on what the users system already has, I may
> need to create one or more levels of this hierarchy.  Do you know if
> there is a way to create this directory structure, as needed, during
> the install step?
>
> Thanks,
> -Steve
>
> On Tue, Mar 26, 2013 at 4:28 PM, Andreas Pakulat <[hidden email] [1]>
> wrote:
>
>> Hi,
>>
>> Am Dienstag, 26. März 2013 schrieb sandrews :
>>
>>> Hi,
>>>
>>> My project is primarily C and C++, but also includes some Python
>>> code.  I
>>> can get everything to build fine, and my CMakeLists.txt files
>>> uses the
>>> INSTALL(TARGETS ...) to so that the user can install the compiled
>>> code to
>>> the proper place.  However, part of the installation is to copy
>>> the Python
>>> files over to their proper places, too, and this isnt working.
>>>
>>> First of all, where should Python files go?  On my Mac, I put
>>> them in
>>> /usr/local/lib/python2.5/site-packages.  Of course though, this
>>> directory
>>> would be different if I had a different version of Python.
>>>  Also, I expect
>>> its different on different platforms.  Does CMake automatically
>>> know where
>>> the Python files should go and, if so, how do I access that?
>>>  (For example,
>>> the INSTALL(TARGETS...) command does know where targets are
>>> supposed to go
>>> for different platforms.)
>>
>>  
>> No, cmake does not know about this as python apps are usually
>> installed with python tools like distutils, setuptools etc. That
>> beingsaid cmake does ship modules to find a python interpreter and
>> that one can be queried about its search path. Along with the
>> install command (using the FILES variant) you can have cmake copy
>> your python code where you want it.
>>
>> However you should think twice before writing cmake code that
>> installs stuff outside thecmake prefix. This is usually unexpected
>> by cmake users, can turn out to be impossible (the user might not be
>> able to writeto thepython install directory) or trip up packaging
>> tools (such as cpack).
>>
>> On the other hand, having python files in something like $HOME/myapp
>> does require an extra environment variable to be set in order for
>> python to find the code there. Thats something that python users are
>> possibly used to though.

I had the same problem when trying to find a proper location for
installing
a SWIG Python module. CMake helps building it but lacks on the
installation
part :-/

I did it like this:
find_package ( PythonLibs REQUIRED )
find_package ( PythonInterp REQUIRED )

if ( PYTHON_VERSION_STRING AND PYTHONLIBS_VERSION_STRING )
   if ( NOT PYTHON_VERSION_STRING VERSION_EQUAL
PYTHONLIBS_VERSION_STRING )
     message ( FATAL_ERROR
             "Version mismatch between python interpreter and libraries"
)
   endif ( NOT PYTHON_VERSION_STRING VERSION_EQUAL
PYTHONLIBS_VERSION_STRING )
endif ( PYTHON_VERSION_STRING AND PYTHONLIBS_VERSION_STRING )

include_directories ( ${PYTHON_INCLUDE_DIRS} )

....target..setup....

execute_process (
   COMMAND ${PYTHON_EXECUTABLE} -c
    "import site, sys; sys.stdout.write(site.PREFIXES[-1])"
   OUTPUT_VARIABLE PYTHON_PREFIX
)
file ( TO_CMAKE_PATH "${PYTHON_PREFIX}" PYTHON_PREFIX )
execute_process (
   COMMAND ${PYTHON_EXECUTABLE} -c
    "import site, sys; sys.stdout.write(site.getsitepackages()[-1])"
   OUTPUT_VARIABLE PYTHON_SITE_DIR
)
file ( TO_CMAKE_PATH "${PYTHON_SITE_DIR}" PYTHON_SITE_DIR )
string ( REGEX REPLACE "^${PYTHON_PREFIX}/" ""
   PYTHON_SITE_DIR "${PYTHON_SITE_DIR}"
)

install ( TARGETS ${SWIG_MODULE_obexftp-python_REAL_NAME}
   LIBRARY
     DESTINATION ${PYTHON_SITE_DIR}
     COMPONENT library
)

install ( FILES ${CMAKE_CURRENT_BINARY_DIR}/obexftp.py
   DESTINATION ${PYTHON_SITE_DIR}
   COMPONENT library
)


My setup is similar for Perl and Ruby SWIG modules as the problem is
the same
there.

HS

--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: Installing Python files

sandrews
Thank you.

At the end, I realized that I could just call the standard Python installer from CMake.  This led to a more reliable install process and also a vastly cleaner CMakeLists.txt file.  My line for installing the Python code is simply:

install(CODE "execute_process(COMMAND python setup.py install -f --prefix=${CMAKE_INSTALL_PREFIX} WORKING_DIRECTORY ../source/libmoleculizer-1.1.2/python-src/language_parser)")

This works well on both of my Macs.  With luck, it will work more broadly, too.  We'll see...

Thanks again for all the help.

-Steve



On Wed, Mar 27, 2013 at 3:32 AM, Hendrik Sattler <[hidden email]> wrote:
Am 2013-03-27 05:46, schrieb Steve Andrews:
Thats very helpful.  Thank you.


My other question was about how to get CMake to create directories at
installation time.  Do you, or someone else, have suggestions about
that?  Despite your advice, I think that I want the "make install"
step to put Python files into
/usr/local/lib/python2.5/site-packages/moleculizer, at least as the
default path.  Depending on what the users system already has, I may

need to create one or more levels of this hierarchy.  Do you know if
there is a way to create this directory structure, as needed, during
the install step?

Thanks,
-Steve

On Tue, Mar 26, 2013 at 4:28 PM, Andreas Pakulat <[hidden email] [1]>
wrote:

Hi,

Am Dienstag, 26. März 2013 schrieb sandrews :

Hi,

My project is primarily C and C++, but also includes some Python
code.  I
can get everything to build fine, and my CMakeLists.txt files
uses the
INSTALL(TARGETS ...) to so that the user can install the compiled
code to
the proper place.  However, part of the installation is to copy
the Python
files over to their proper places, too, and this isnt working.


First of all, where should Python files go?  On my Mac, I put
them in
/usr/local/lib/python2.5/site-packages.  Of course though, this
directory
would be different if I had a different version of Python.
 Also, I expect
its different on different platforms.  Does CMake automatically

know where
the Python files should go and, if so, how do I access that?
 (For example,
the INSTALL(TARGETS...) command does know where targets are
supposed to go
for different platforms.)

 
No, cmake does not know about this as python apps are usually
installed with python tools like distutils, setuptools etc. That
beingsaid cmake does ship modules to find a python interpreter and
that one can be queried about its search path. Along with the
install command (using the FILES variant) you can have cmake copy
your python code where you want it.

However you should think twice before writing cmake code that
installs stuff outside thecmake prefix. This is usually unexpected
by cmake users, can turn out to be impossible (the user might not be
able to writeto thepython install directory) or trip up packaging
tools (such as cpack).

On the other hand, having python files in something like $HOME/myapp
does require an extra environment variable to be set in order for
python to find the code there. Thats something that python users are
possibly used to though.

I had the same problem when trying to find a proper location for installing
a SWIG Python module. CMake helps building it but lacks on the installation
part :-/

I did it like this:
find_package ( PythonLibs REQUIRED )
find_package ( PythonInterp REQUIRED )

if ( PYTHON_VERSION_STRING AND PYTHONLIBS_VERSION_STRING )
  if ( NOT PYTHON_VERSION_STRING VERSION_EQUAL PYTHONLIBS_VERSION_STRING )
    message ( FATAL_ERROR
            "Version mismatch between python interpreter and libraries" )
  endif ( NOT PYTHON_VERSION_STRING VERSION_EQUAL PYTHONLIBS_VERSION_STRING )
endif ( PYTHON_VERSION_STRING AND PYTHONLIBS_VERSION_STRING )

include_directories ( ${PYTHON_INCLUDE_DIRS} )

....target..setup....

execute_process (
  COMMAND ${PYTHON_EXECUTABLE} -c
        "import site, sys; sys.stdout.write(site.PREFIXES[-1])"
  OUTPUT_VARIABLE PYTHON_PREFIX
)
file ( TO_CMAKE_PATH "${PYTHON_PREFIX}" PYTHON_PREFIX )
execute_process (
  COMMAND ${PYTHON_EXECUTABLE} -c
        "import site, sys; sys.stdout.write(site.getsitepackages()[-1])"
  OUTPUT_VARIABLE PYTHON_SITE_DIR
)
file ( TO_CMAKE_PATH "${PYTHON_SITE_DIR}" PYTHON_SITE_DIR )
string ( REGEX REPLACE "^${PYTHON_PREFIX}/" ""
  PYTHON_SITE_DIR "${PYTHON_SITE_DIR}"
)

install ( TARGETS ${SWIG_MODULE_obexftp-python_REAL_NAME}
  LIBRARY
    DESTINATION ${PYTHON_SITE_DIR}
    COMPONENT library
)

install ( FILES ${CMAKE_CURRENT_BINARY_DIR}/obexftp.py
  DESTINATION ${PYTHON_SITE_DIR}
  COMPONENT library
)


My setup is similar for Perl and Ruby SWIG modules as the problem is the same
there.

HS


--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake


--

Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake