Cross compile project using pybind11 for windows on Linux using MinGW

[rhvarrier]

I have a C++ library for which I'm using pybind11 to generate Python bindings. Currently I'm compiling on Linux x86 host for the same target using gcc and everything works fine.
I want to port my library (which only depends on C ++ std libraries) to Windows and I was to keep my current Linux development environment and cross compile the library for Windows using MinGW (from MXE). However when I try to build the python bindings using pybind11, the compilation fails as MinGW cannot find a compatible Python library.
My question is as follows:

1. Has anyone else had a similar issue and if so is there a workaround. I tried compiling CPython using MinGW but it seems quite tedious and not well supported by current Python versions (> 3.9)
2. Is there a way that pybind11 generated module does not link to Python at all (I could just provide the necessary header files) But at runtime (i.e. when a python script that imports the module gets interpreted), it automatically loads the Python DLL (the one that ships with Python and was most probably compiled with MSVC). If this works I could just distribute by python module along with a compatible Python DLL
   If there is a 3rd option I would be most welcome as well (N.B. I cannot compile on Windows and I need to keep my a cross compilation environment (not necessarily MXE)

[lvk88]

Hey there, I know this is a very old answer question, but just in case someone needs to do this: I recently had to do something similar, it was very difficult get it running, but at the end it worked out. CMake needs a lot of handholding to understand that we don't want to find the python libraries from the host system. I had to do the following:

• First, you need to download the Windows version of the Python interpreter and libraries using NuGet. As surprising as it sounds, NuGet is the officially recommended way on Python website to get an arbitrary version of Python for CI purposes. To get nuget, you need mono-devel first, because NuGet is a dotnet application. So something like:

apt install mono-devel
wget https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
mono nuget.exe install python -Version 3.13.0

• Second, you need to tell CMake a a lot of information about the freshly downloaded Python library so that it really finds it. For me, this monster worked out (assuming that the NuGet step downloaded Python in the same folder where CMakeLists.txt of your project resides):

cmake \
 -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++ \
 -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc \
 -DCMAKE_SYSTEM_NAME=Windows \
+ -DCMAKE_FIND_ROOT_PATH="/usr/x86_64-w64-mingw32;$PWD/python.3.13.0" \
 -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY \
 -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY \
 -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \
+ -DPython_INCLUDE_DIR=$PWD/python.3.13.0/tools/include \
+ -DPython_LIBRARY=$PWD/python.3.13.0/tools/python313.dll \
+ -DPYBIND11_USE_CROSSCOMPILING=ON \
+ -DPYTHON_MODULE_EXT_SUFFIX=".cp313-win_amd64.pyd" \
+ -DPYTHON_MODULE_EXTENSION=".cp313-win_amd64.pyd" \
 -S . -B build/

The green lines represent those that I had to add because of cross-compiling a pybind11 module.

• Even if you do all this, CMake might still fail to find Python properly, because somewhere deeply ther is still a find_program happening. So in my CMake project I had to set CMAKE_FIND_ROOT_PATH_MODE_PROGRAM to ONLY before including pybind11 and set it back to NEVER after pybind11 detection has settled on a Python. Something like this

+ if(CMAKE_CROSSCOMPILING)
+  set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ONLY)
+ endif(CMAKE_CROSSCOMPILING)

FetchContent_MakeAvailable(pybind11)

+if(CMAKE_CROSSCOMPILING)
+  set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
+endif(CMAKE_CROSSCOMPILING)

So now, cmake will manage to find the right python and not the one on the host system, and building will also work out.

• However, if you copy the pyd module that you just built to a Windows system and try to import it, python will still complain that it cannot find some DLL files. This is because the module was built with mingw and so it will need a couple of runtime dll-s from mingw in the same folder where the pyd file is to work, most notably libgcc_s_seh-1.dll libstdc++-6.dll libwinpthread-1.dll. So you either manually copy them next to your pyd or introduce an extra cmake rule that does this copying for example at install time.

This is a scaringly complicated process. If it helps, I created a repo where all this is demonstrated step by step, including a GitHub workflow that builds a pyd module on Debian and runs the tests on this module on Windows. There is also a GitLab equivalent in the repo.

See the repo here:

https://github.com/lvk88/mingw-pipeline-tests

I hope this helps.