[HOWTO] Easily Cross-Compile CMake (with xstatic)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[HOWTO] Easily Cross-Compile CMake (with xstatic)

Zach van Rijn
Hi Friends,


I have, until recently, been under the impression that CMake is
rather difficult (or impossible?) to cross-compile correctly. I
believe I have devised a sane method of doing so. In addition to
being simple, the output binaries are fully static, so they may
be transferred to any compatible system without dependencies. It
has been tested on CMake version 3.12.3, on 48 platforms, of
which 19 do not build without patches (contribs welcome!)

This method does _not_ require having a recent version of CMake
on the build system, nor does it require any other dependencies
except GNU 'make', 'gcc', 'g++', and some tool like wget/curl to
fetch the source code. We leverage "bootstrap" process products.

The primary motivation for wanting to cross-compile CMake itself
is two-fold: (a) Target systems do not always have compilers for
C++, and not all CMake-based projects require a C++ compiler. If
the system does not have a sufficiently-recent CMake, and (b) An
informative guide such as this might be of interest to the CMake
and/or other communities.

In addition, official binary distributions include only x86_64
Linux and Darwin, and x86(_64) Win32. What about ARM, MIPS, PPC,
others? The obvious solution is to build natively in QEMU, a VM,
or on the target platform if one is fortunate to have this. Some
Linux distributions [1] [2] [3] provide a few such packages.

Note that, this is highly experimental and there are some kinks
that must be resolved before all platforms are supported.


Introduction
------------

Documentation for existing methods [4] is sparse, complicated,
and either incomplete or incorrect (usually stemming from a user
posting a question about how to accomplish this, and the thread
goes stale). In addition to the mailing list, queries to Google,
StackOverflow [5], etc. are often fruitless.

Below is a method that any user should be able to follow in
order to produce the following:

  * Static CMake ('cmake', 'cpack', 'ctest') binaries that run
    almost anywhere without dependencies;

  * The associated 'doc/' and 'share/' directories (some of them
    contain files necessary for proper CMake execution); and

  * NOT Windows-friendly or MinGW-based binaries. This fails due
    to a "fatal error: byteswap.h: No such file or directory"
    problem in one of the self-contained (LZMA?) CMake deps, and
    I suspect it is trivial to work around this but I've not yet
    had the time to try.

The above is achieved using the 'musl' C standard library [6], a
minimal and lightweight implementation designed for static
builds. It may also be possible using 'glibc' if static versions
are available. This has not been tested.

 **READ AND UNDERSTAND ALL STEPS BEFORE MODIFYING YOUR SYSTEM!**


Requirements
------------

This method is not dependent upon, but is significantly easier
to perform, if you have access to a Linux-like system that has
some sort of shell (e.g., 'dash' or 'bash'). We make use of
symbolic links and other common *nix commands. Please do not
download, copy/paste, or execute anything that you are uncertain
of or do not understand fully what it is or does.

This process involves *breaking* any existing native (BUILD)
toolchain and it is therefore recommended you perform these
steps in a chroot, container, or virtualized ephemeral space.

You will also want to empty '/usr/local/' prior to starting, or
wisely choose a different installation directory later in this
process to avoid contamination and/or system breakage.


(1) A cross-compilation toolchain, which runs on BUILD and
    produces code for HOST. In this example, I use toolchains
    from musl.cc [7]. These can produce static code.

      - BUILD = x86_64-linux-musl  (it's what I'm running)

      -  HOST = aarch64-linux-musl (anything except MinGW)

    From the musl.cc website, this is a '-cross' toolchain.
    You may pick any flavor you like, or search online for other
    similar toolchains if your platform is not available here. A
    toolchain which produces static code is ideal, but should be
    compatible with your target (HOST) platform in either case.


(2) A native-compilation toolchain, which is required only for
    the "bootstrap" phase of the CMake build process. This does
    not need to be from the same site, but that's how I tested.
    In this example, you only need to be able to bootstrap.

      - BUILD = x86_64-linux-musl (yours can be glibc or other)

      -  HOST = x86_64-linux-musl (you must be able to execute)

    From the musl.cc website, this is a '-native' toolchain.
    You do not need this if you have a $CC and $CXX that works
    properly (trial and error, but the musl.cc toolchains have
    been tested). They need not produce static code.


(3) A modern-ish version of GNU 'make' (I don't know what the
    minimum is, but I'm using GNU Make 4.2.1). Others may work;
    this probably varies by CMake version. Check their docs?


(4) Patience, and understanding that this process may not work
    on your platform(s), and that you assume all risks/etc.


Procedure
---------

The steps detailed below are also encoded in a shell script that
is part of the 'xstatic' project [8]. This is how these methods
were tested, and you may find these scripts convenient. Below is
an annotated procedure (including caveats) when cross-compiling
CMake. Please follow closely and work in an isolated directory.

If you have Docker, you can download pre-packaged versions of
the cross-compiler toolchains from here [9] and build inside.


(1) Set the following environment variables ($WORK is where you
    will build CMake; make sure it exists) in your shell:

WORK=/tmp                       # scratch directory inside cont.
CONF=59e2ce0e6b46               # config.sub replacement (git)
XMCM=aarch64-linux-musl         # HOST  triple
HVER=x86_64-linux-musl          # BUILD triple
VCMK=3.12.3                     # CMake version to build


(2) Download a musl.cc native (BUILD) compiler toolchain. You do
    not need to do this if you have a compatible compiler, but I
    strongly recommend it. We will *BREAK* this compiler later.

$ cd ${WORK}
$ curl -so ${HVER}-native.tgz https://musl.cc/${HVER}-native.tgz
$ tar -xf  ${HVER}-native.tgz
$ rm       ${HVER}-native.tgz


(3) Download a musl.cc cross (HOST) compiler toolchain. This is
    also not required, but is almost guaranteed necessary unless
    you have a musl-based (or static glibc-based, etc.) one. The
    cross toolchains on this website are intended for use on
    x86_64 Linux systems (i686 ones are available, too).

$ cd ${WORK}
$ curl -so ${XMCM}-cross.tgz  https://musl.cc/${XMCM}-cross.tgz
$ tar  -xf ${XMCM}-cross.tgz
$ rm       ${XMCM}-cross.tgz

    The Docker images are presently configured like so. DO NOT
    PERFORM THESE STEPS OUTSIDE OF A CONTAINER OR VM! These will
    likely break your existing system compiler(s). Only do this
    if you do not have a system compiler.

# cd ${WORK}
# cd       ${XMCM}-cross
# rsync -rLq . /                # WRITES TO ROOT FILESYSTEM!
# cd /bin
# rename "${XMCM}-" "" *

    This causes 'gcc' to refer to the cross (HOST) compiler, and
    '${WORK}/${HVER}-native/bin/gcc' to the native (BUILD) one.

    Instead, you can modify your $PATH variable to point to the
 
  correct compiler binaries, at '${WORK}/${HVER}-native/bin/'.

    One other (possibly) important detail is that I've wrapped
    the three compilers ('gcc', 'g++', and 'gfortran') like so:

for k in g++ gcc gfortran; do
    p=$(which ${k})     || true
    [ ! "x${p}" = "x" ] || continue;
    [ ! -e ${p}.orig  ] || continue;
    mv ${p}  ${p}.orig
    echo >   ${p} '#!/bin/sh'
    echo >>  ${p} "${k}.orig \${@} -static --static -g0 -s -Os"
    chmod +x ${p}
done

    The reason is simple: to force, absolutely, that all object
    code produced by the compilers to be static, stripped, and
    optimized for size. Not tested without it. Season to taste.


(4) Download and unpack the CMake source code from a tarball:

$ curl -so cmake-${VCMK}.tar.gz \
    https://cmake.org/files/v$(echo ${VCMK} \
        | awk -F'[.]' '{print $1 "." $2}')/cmake-${VCMK}.tar.gz
$ tar  -xf cmake-${VCMK}.tar.gz
$ rm       cmake-${VCMK}.tar.gz
$ cd       cmake-${VCMK}


(5) "Bootstrap" CMake using the native (BUILD) compiler (this is
    one long command):

$ ${WORK}/${HVER}-native/bin/gcc --version && \
CC="${WORK}/${HVER}-native/bin/gcc" \
CFLAGS="-static --static" \
CXX="${WORK}/${HVER}-native/bin/g++" \
CXXFLAGS="-static --static" \
    ./bootstrap --parallel=$(nproc)


(6) Overwrite (break) the native (BUILD) compiler by creating
    symbolic links to is components. Until further modifications
    are made to CMake itself, this is the easiest way to ensure
    that the correct compilers are picked up.

    In this example, it is expected that "$(which gcc)" refers
    to the cross (HOST) compiler, and the native (BUILD) one is:
    '${WORK}/${HVER}-native/bin/gcc'. Adjust paths accordingly.

$ ln -sf $(which g++  ) ${WORK}/${HVER}-native/bin/g++
$ ln -sf $(which gcc  ) ${WORK}/${HVER}-native/bin/gcc
$ ln -sf $(which ld   ) ${WORK}/${HVER}-native/bin/ld
$ ln -sf $(which strip) ${WORK}/${HVER}-native/bin/strip


(7) Build the "real" CMake binaries using the cross (HOST)
    compilers (which is now accessible via the location of the
    old (now broken) native (BUILD) compilers. The previous step
    of "bootstrapping" already configured CMake.

$ make -j$(nproc)


(8) Move the "real" CMake binary elsewhere (for now) as we need
    to use a CMake binary that runs on the BUILD system, but the
    one we just compiled only runs on HOST. So, we'll utilize a
    special version of CMake (the one built during "bootstrap")
    to complete the next step.

$ mv bin/cmake cmake.real # symlink is disallowed!
$ cp Bootstrap.cmk/cmake bin/cmake


(9) The typical "make install" procedure, frustratingly requires
    a functioning CMake executable, and its path is hard-coded
    to be the one we built in the previous step. The "bootstrap"
    CMake binary will not work anywhere outside of the project
    directory, but it will suffice just for installation.

$ make install
$ mv cmake.real /usr/local/bin/cmake


Now, all files contained in '/usr/local/' (or wherever you may
choose to $DESTDIR it to) are built statically for the HOST. You
no longer have a functioning BUILD compiler unless you renamed
the files before overwriting the executables with symbolic links
to your HOST compiler.


Testing
-------

Copy these files somewhere on a compatible system. They do not
need to be moved to '/usr/local/' at all; your $HOME directory
is sufficient.


$ cd /usr/local/bin
$ file *
cmake: ELF 64-bit LSB  executable, ARM aarch64, version 1
(SYSV), statically linked, stripped
cpack: ELF 64-bit LSB  executable, ARM aarch64, version 1
(SYSV), statically linked, stripped
ctest: ELF 64-bit LSB  executable, ARM aarch64, version 1
(SYSV), statically linked, stripped


$ ls -l
total 13692
-rwxr-xr-x 1 zv zv 4518992 Oct 25 22:15 cmake
-rwxr-xr-x 1 zv zv 4510296 Oct 25 22:15 cpack
-rwxr-xr-x 1 zv zv 4982384 Oct 25 22:15 ctest


$ ./cmake --version
cmake version 3.12.3

CMake suite maintained and supported by Kitware
(kitware.com/cmake).


It is also possible to use these binaries within QEMU if there
is a compatible user-level one available:

$ qemu-aarch64 ./cmake --version
cmake version 3.12.3

CMake suite maintained and supported by Kitware
(kitware.com/cmake).


One example of a CMake-based C-only (no C++) project is the
SUNDIALS library from LLNL [10]. The system on which I am
testing this does not have CMake installed otherwise.

$ which cmake
/home/zv/aarch64-linux-musleabi/bin/cmake
$ SVER=3.2.1
$ wget https://computation.llnl.gov/projects/sundials/download/\
sundials-${SVER}.tar.gz
$ tar -xf sundials-${SVER}.tar.gz
$ rm      sundials-${SVER}.tar.gz
$ mkdir   sundials-${SVER}/build
$ cd      sundials-${SVER}/build
$ cmake ..
$ make -j$(nproc)


Download
--------

Pre-built (again using 'xstatic') binaries are available for
testing. Please verify the checksums and build your own from
source before using them in any non-testing environment.

    https://xstatic.musl.cc/cmake/

The following platforms DID NOT build successfully. Your mileage
may vary (depending on compiler, C/C++ libraries, etc.) so
please report any successful combinations you may discover:

  * i686/x86_64 PE32 using MinGW; I suspect adding this:

        -DCMAKE_SYSTEM_NAME:STRING="Windows"

    might resolve the issue and I'll test in due course.

  * MicroBlaze (LE/BE); probably needs -fPIC or a patch:

        BFD (GNU Binutils) 2.31.1 assertion fail
        ../../src_binutils/bfd/elf32-microblaze.c:1542

  * microblaze-linux-musl
  * microblazeel-linux-musl
  * mips-linux-musl
  * mips-linux-musln32sf
  * mips-linux-muslsf
  * mips64-linux-musln32
  * mips64-linux-musln32sf
  * mips64el-linux-musln32
  * mips64el-linux-musln32sf
  * mipsel-linux-musl
  * mipsel-linux-musln32
  * mipsel-linux-musln32sf
  * mipsel-linux-muslsf
  * or1k-linux-musl
  * riscv32-linux-musl
  * riscv64-linux-musl
  * x86_64-linux-musl (odd, but OK, same kind of error)

These may actually be issues with the above method, or with the
provided toolchains, and I will investigate these further when
time permits. It seems, however, that things are mostly working.


Regards,

ZV


References
----------

[1] https://packages.debian.org/search?keywords=cmake

[2] https://pkgs.alpinelinux.org/packages?name=cmake&branch=edge

[3] https://packages.ubuntu.com/search?keywords=cmake

[4] <a href="https://www.mail-archive.com/search?q=cross-compile&l=cmake%">https://www.mail-archive.com/search?q=cross-compile&l=cmake%
40cmake.org (not quite 'porting CMake to other platforms') and
https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/Cross
Compiling (mainly for CMake-project developers)

[5] https://stackoverflow.com/search?q=cross-compile+cmake

[6] https://www.musl-libc.org/

[7] https://musl.cc/

[8] https://git.zv.io/me/xstatic

[9] https://hub.docker.com/r/muslcc/x86_64/tags/

[10] https://computation.llnl.gov/projects/sundials/

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: [HOWTO] Easily Cross-Compile CMake (with xstatic)

Rolf Eike Beer
Am 2018-10-26 10:04, schrieb Zach van Rijn:

> Hi Friends,
>
>
> I have, until recently, been under the impression that CMake is
> rather difficult (or impossible?) to cross-compile correctly. I
> believe I have devised a sane method of doing so. In addition to
> being simple, the output binaries are fully static, so they may
> be transferred to any compatible system without dependencies. It
> has been tested on CMake version 3.12.3, on 48 platforms, of
> which 19 do not build without patches (contribs welcome!)

Hi Zach,

as someone who does crosscompiling for a living I have read your post,
even if I have never seen a need for what you are doing. It's an
interesting approach, although it looks pretty brute force. Not that I
have not done things like that myself (has anyone tried to compile a
recent udev without all the systemd things around it and without
packaging the dependencies only needed for the systemd parts before? you
know what I mean.).

I had an old cross-build recipe for CMake around and decided to clean
the dust away and see if it still works. Well, it did not, but that was
just because the embedded curl tries to do some try_run() things that do
not work. Since I already have a cross-curl library at hand I decided to
just use that, and things worked.

So, the 2 things that I do not understand why you need them at all:

-why do the bootstrap thing and move compilers? Just build a recent
CMake with your host toolchain, package that, and drop it into the build
container. The need for the host compiler should be gone at this point

-why do you need to overwrite the compiler with the target compiler?
Just drop it anywhere and tell CMake with CC and CXX variables where it
is. Or better, pass it a toolchain file like mine:

=== toolchain.cmake ===
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)

# specify the cross compiler
set(CMAKE_C_COMPILER /opt/emlix/test/bin/arm-unknown-linux-gnueabi-gcc)
set(CMAKE_CXX_COMPILER
/opt/emlix/test/bin/arm-unknown-linux-gnueabi-g++)

# where is the target environment
set(CMAKE_FIND_ROOT_PATH /opt/emlix/test/sysroot)
set(CMAKE_SYSROOT /opt/emlix/test/sysroot)

# search for programs in the build host directories
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
===

I'm pretty sure you may find some rough edges, especially if you do not
want to have the dependencies pre-build but use the embedded copies and
do static builds with them. But otherwise you should hopefully be able
to simplify your setup a lot.

Eike
--
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: [HOWTO] Easily Cross-Compile CMake (with xstatic)

Zach van Rijn
On Fri, 2018-10-26 at 11:09 +0200, Rolf Eike Beer wrote:
> ...

Hi Eike,

Thank you for writing. I agree that my approach is a bit brutish
and aim to revise/reduce it to a minimum example. My goal was to
describe a process that is uniform between compilers, easily
automated, and less likely to break between CMake releases.

> So, the 2 things that I do not understand why you need them at
> all:
>
> -why do the bootstrap thing and move compilers? Just build a
> recent CMake with your host toolchain, package that, and drop
> it into the build container. The need for the host compiler
> should be gone at this point

While it would make sense to build a complete (sufficiently-
recent) CMake that performs the final build, I have not had
consistent success (using the same exact compilers and systems)
in doing so. For example, even specifying all of the settings as
in your 'toolchain.cmake' file (below in this email), omitting
the 'CMAKE_SYSTEM_NAME=Linux' part:

...
cmake-3.12.3/Utilities/cmcurl/lib/strerror.c:32:6: error: #error
"strerror_r MUST be either POSIX, glibc or vxworks-style"
...

and the build fatally aborts. If we add this back,

...
CMake Error: TRY_RUN() invoked in cross-compiling mode, please
set the following cache variables appropriately:
...

which means, potentially, that each future release of CMake may
require updates to the script. I'm not sure if there's a way to
cache all of these variables at once?

CMake Error: TRY_RUN() invoked in cross-compiling mode, please
set the following cache variables appropriately:
   KWSYS_LFS_WORKS (advanced)                                 
   KWSYS_LFS_WORKS__TRYRUN_OUTPUT (advanced) 

CMake Error: TRY_RUN() invoked in cross-compiling mode, please
set the following cache variables appropriately:
   HAVE_FSETXATTR_5
(advanced)                                         
   HAVE_FSETXATTR_5__TRYRUN_OUTPUT (advanced) 

CMake Error: TRY_RUN() invoked in cross-compiling mode, please
set the following cache variables appropriately:
   HAVE_GLIBC_STRERROR_R
(advanced)                               
   HAVE_GLIBC_STRERROR_R__TRYRUN_OUTPUT (advanced)

CMake Error: TRY_RUN() invoked in cross-compiling mode, please
set the following cache variables appropriately:
   HAVE_POSIX_STRERROR_R (advanced)                            
   HAVE_POSIX_STRERROR_R__TRYRUN_OUTPUT (advanced) 

CMake Error: TRY_RUN() invoked in cross-compiling mode, please
set the following cache variables appropriately:
   HAVE_POLL_FINE_EXITCODE
(advanced)                                
   HAVE_POLL_FINE_EXITCODE__TRYRUN_OUTPUT (advanced) 

So it is certainly possible to do this... assuming you are able
to obtain a recent copy of CMake for your build platform:

# In an empty 'build/' directory in the root of CMake source:

$ cmake ..                                                     \
    -DCMAKE_SYSTEM_NAME=Linux                                  \
    -DCMAKE_C_COMPILER=/bin/gcc                      `# cross` \
    -DCMAKE_C_FLAGS="-static --static -g0 -s -Os"              \
    -DCMAKE_CXX_COMPILER=/bin/g++                    `# cross` \
    -DCMAKE_CXX_FLAGS="-static --static -g0 -s -Os"            \
    -DCMAKE_FIND_ROOT_PATH=/armv7l-linux-musleabihf  `# cross` \
    -DCMAKE_SYSROOT=/armv7l-linux-musleabihf         `# cross` \
    -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER                  \
    -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY                   \
    -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY                   \
    -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY                   \
    -DKWSYS_LFS_WORKS=ON                                       \
    -DKWSYS_LFS_WORKS__TRYRUN_OUTPUT=""                        \
    -DHAVE_FSETXATTR_5=ON                                      \
    -DHAVE_FSETXATTR_5__TRYRUN_OUTPUT=""                       \
    -DHAVE_GLIBC_STRERROR_R=OFF               `# because musl` \
    -DHAVE_GLIBC_STRERROR_R__TRYRUN_OUTPUT=""                  \
    -DHAVE_POSIX_STRERROR_R=ON                `# because musl` \
    -DHAVE_POSIX_STRERROR_R__TRYRUN_OUTPUT=""                  \
    -DHAVE_POLL_FINE_EXITCODE=ON                               \
    -DHAVE_POLL_FINE_EXITCODE__TRYRUN_OUTPUT=""

...
CMake Warning:
  Manually-specified variables were not used by the project:

    KWSYS_LFS_WORKS__TRYRUN_OUTPUT
...

$ make -j$(nproc)
$ make install

$ file /usr/local/bin *
cmake: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV),
statically linked, stripped
cpack: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV),
statically linked, stripped
ctest: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV),
statically linked, stripped

>
> -why do you need to overwrite the compiler with the target
> compiler? Just drop it anywhere and tell CMake with CC and CXX
> variables where it is. Or better, pass it a toolchain file
> like mine:

Specifically because it would either require defining all of the
above variables, including toolchain path(s), or running some
sort of script or command like 'sed' to attempt to replace all
hard-coded occurrences of the original toolchain. You can see
that there are many:

$ cd ~/cmake-3.12.3/build
$ grep -r "/bin/g++" . | wc -l
1247

(This 'grep' command actually takes longer than building CMake!)

This would be further complicated by having to escape toolchain
paths if they are not in similar locations, and could be messy.

>
> === toolchain.cmake ===
> set(CMAKE_SYSTEM_NAME Linux)
> set(CMAKE_SYSTEM_PROCESSOR arm)
>
> # specify the cross compiler
> set(CMAKE_C_COMPILER /opt/emlix/test/bin/arm-unknown-linux-
> gnueabi-gcc)
> set(CMAKE_CXX_COMPILER 
> /opt/emlix/test/bin/arm-unknown-linux-gnueabi-g++)
>
> # where is the target environment
> set(CMAKE_FIND_ROOT_PATH /opt/emlix/test/sysroot)
> set(CMAKE_SYSROOT /opt/emlix/test/sysroot)
>
> # search for programs in the build host directories
> set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
> # for libraries and headers in the target directories
> set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
> set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
> set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
> ===
>
> I'm pretty sure you may find some rough edges, especially if
> you do not want to have the dependencies pre-build but use the
> embedded copies and do static builds with them. But otherwise
> you should hopefully be able to simplify your setup a lot.

Once all this is done, how can this be easily automated? Pasting
in the appropriate 'CMAKE_SYSTEM_NAME' (and others) via some
script? Is it possible to determine the 'HAVE_' and '_WORKS'
variables ahead of time (which ones are actually need TRY_RUN)?

In summary, while I agree my original method could be
simplified, I propose it for the following reasons:

(1) Avoid building two full CMake binaries (one for build system
    and one for target host system)

(2) Avoid breakage between CMake versions, if new tests are
    are added, as script automation would become difficult

(3) As an alternative to the "standard" approach which you
    mention, as some users may have difficulty in determining
    exactly how to populate these variables.


> Eike
> -- 

The approach outlined in this email (based off of yours) might
work for Windows (MinGW) builds. I will try it for the remaining
architectures which did not work with the original approach.


ZV

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
Reply | Threaded
Open this post in threaded view
|

Re: [HOWTO] Easily Cross-Compile CMake (with xstatic)

Rolf Eike Beer
Zach van Rijn wrote:

> On Fri, 2018-10-26 at 11:09 +0200, Rolf Eike Beer wrote:
> > ...
>
> Hi Eike,
>
> Thank you for writing. I agree that my approach is a bit brutish
> and aim to revise/reduce it to a minimum example. My goal was to
> describe a process that is uniform between compilers, easily
> automated, and less likely to break between CMake releases.
>
> > So, the 2 things that I do not understand why you need them at
> > all:
> >
> > -why do the bootstrap thing and move compilers? Just build a
> > recent CMake with your host toolchain, package that, and drop
> > it into the build container. The need for the host compiler
> > should be gone at this point
>
> While it would make sense to build a complete (sufficiently-
> recent) CMake that performs the final build, I have not had
> consistent success (using the same exact compilers and systems)
> in doing so. For example, even specifying all of the settings as
> in your 'toolchain.cmake' file (below in this email), omitting
> the 'CMAKE_SYSTEM_NAME=Linux' part:
>
> ...
> cmake-3.12.3/Utilities/cmcurl/lib/strerror.c:32:6: error: #error
> "strerror_r MUST be either POSIX, glibc or vxworks-style"
> ...
That's exactly what I meant with:

> > Well, it did not, but that was just because the embedded curl tries to do
> > some try_run() things that do not work. Since I already have a cross-curl
> > library at hand I decided to just use that, and things worked.

> and the build fatally aborts. If we add this back,
>
> ...
> CMake Error: TRY_RUN() invoked in cross-compiling mode, please
> set the following cache variables appropriately:
> ...

You can instead just pass the expected codes, you just have to pass them as
cache variables using "-C cache_file".

> # In an empty 'build/' directory in the root of CMake source:
>
> $ cmake ..                                                     \
>     -DCMAKE_SYSTEM_NAME=Linux                                  \
>     -DCMAKE_C_COMPILER=/bin/gcc                      `# cross` \
>     -DCMAKE_C_FLAGS="-static --static -g0 -s -Os"              \
>     -DCMAKE_CXX_COMPILER=/bin/g++                    `# cross` \
>     -DCMAKE_CXX_FLAGS="-static --static -g0 -s -Os"            \
>     -DCMAKE_FIND_ROOT_PATH=/armv7l-linux-musleabihf  `# cross` \
>     -DCMAKE_SYSROOT=/armv7l-linux-musleabihf         `# cross` \
>     -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER                  \
>     -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY                   \
>     -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY                   \
>     -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY                   \
>     -DKWSYS_LFS_WORKS=ON                                       \
>     -DKWSYS_LFS_WORKS__TRYRUN_OUTPUT=""                        \
>     -DHAVE_FSETXATTR_5=ON                                      \
>     -DHAVE_FSETXATTR_5__TRYRUN_OUTPUT=""                       \
>     -DHAVE_GLIBC_STRERROR_R=OFF               `# because musl` \
>     -DHAVE_GLIBC_STRERROR_R__TRYRUN_OUTPUT=""                  \
>     -DHAVE_POSIX_STRERROR_R=ON                `# because musl` \
>     -DHAVE_POSIX_STRERROR_R__TRYRUN_OUTPUT=""                  \
>     -DHAVE_POLL_FINE_EXITCODE=ON                               \
>     -DHAVE_POLL_FINE_EXITCODE__TRYRUN_OUTPUT=""
>
> ...
> CMake Warning:
>   Manually-specified variables were not used by the project:
>
>     KWSYS_LFS_WORKS__TRYRUN_OUTPUT
You need to pass things like CMAKE_SYSTEM_NAME in the toolchain file in order
to get the cross-build mode properly initialized, passing them on the
commandline is not enough.

> Specifically because it would either require defining all of the
> above variables, including toolchain path(s), or running some
> sort of script or command like 'sed' to attempt to replace all
> hard-coded occurrences of the original toolchain. You can see
> that there are many:

No, you start in a clean build directory and pass the initial CMake invocation
the toolchain file with -D CMAKE_TOOLCHAIN_FILE=toolchain-arm.cmake. CMake
will then know only this compiler, and will find the other programs using the
same triplet. Have one build dir per target, and one toolchain file per
target. Maybe it's only one toolchain file that you need, when they differ
only in the target triplet, which can then be placed e.g. in an environment
variable before the first build.

> In summary, while I agree my original method could be
> simplified, I propose it for the following reasons:
>
> (1) Avoid building two full CMake binaries (one for build system
>     and one for target host system)

You do, bootstrapping builds a CMake binary. Not a full-featured one, but
still. And you can reuse them for every target.

> (2) Avoid breakage between CMake versions, if new tests are
>     are added, as script automation would become difficult

These should not differ between patch releases, and the differences between
minor versions should also be pretty minimal.

> (3) As an alternative to the "standard" approach which you
>     mention, as some users may have difficulty in determining
>     exactly how to populate these variables.

"-C"

Eike
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

signature.asc (201 bytes) Download Attachment