Skip to content

Add multi-compiler support and cross-platform build automation#175

Open
Scanframe wants to merge 16 commits into
OHF-Voice:mainfrom
Scanframe:main
Open

Add multi-compiler support and cross-platform build automation#175
Scanframe wants to merge 16 commits into
OHF-Voice:mainfrom
Scanframe:main

Conversation

@Scanframe

@Scanframe Scanframe commented Feb 7, 2026

Copy link
Copy Markdown

Description

This pull request adds comprehensive multi-compiler and cross-platform build support to enable building the Piper library and a CLI test application with various toolchains across different platforms.
This allows developers to test builds across all supported toolchains from a single Linux machine native or via Wine, Docker, Docker+Wine and when needed on Windows.
Minor changes have been made to the C++ source fixing some shortcomings for other toolchains and a serious MSVC only bug.

See it in action on YouTube
The repository Piper TTS Qt-Plugin is depending on these PR's changes.

How To

Simply run this in Linux with docker installed:

Build and Test Linux using its host architecture

./build.py docker -- --build --test gnu-debug

Build and Test Linux using architecture aarch64 (cross-compiled)

./build.py docker -- --build --test ga-debug

Build and Test for Windows using GNU for Windows (cross-compiled)

The test is executed in Wine.

./build.py docker -- --build --test gw-debug

Build and Test for Windows using MSVC-2022 compiler

Compiler and the test are running in Linux/Wine.

./build.py docker -- wine -- --build --test msvc-debug

The output.wav is the testresult located the bin/lnx64-<compiler> or bin/win64-<compiler> directory.

Toolchains Available for Building

This a list of all available toolchains and how they are referenced in the CMake configuration.:

  • gnu - GNU GCC Linux x86_64 or aarch64 which depends on the build host
  • mingw - MinGW-w64 (GNU GCC for Windows x86_64)
  • gw - GNU GCC cross-compiler for Windows on Linux x86_64
  • ga - Aarch64 GNU GCC cross-compiler on Linux x86_64
  • msvc - Microsoft Visual C++ 2022 x86_64

Platform Support:

  • Linux Native - Direct execution on the host OS (x86_64 or aarch64)
  • Windows Native - Direct execution on the host OS (x86_64)
  • Linux Wine - Windows emulation layer on Linux (x86_64)
  • Linux Docker - Containerized build environment (x86_64)
  • Linux Docker + Wine - Wine running inside Docker (x86_64)

Key Changes

  • Fixed MSVC-specific issues as byte packing and global instance declaration in a header that caused a segmentation fault.
  • Added CMakePresets.json with toolchain-specific configurations and separate build directories
  • Added build.py automation script with build.ini configuration supporting all 11 tool/platform combinations
  • Added toolchain-specific CMake files for each compiler
  • Implemented run-executable.cmake helper script for cross-platform test execution
    using the environment and variables set in the CMakePresets.json file.
  • Added a test application using a piper-cli application with CTest.
  • The main CMakeLists.txt with CMakePresets.json suitable for working with JetBrains/CLion or VSCode.

CLion Autmatic Toolchain Selection

To have the correct toolchain selected in JetBrain's CLion the vendor section is used.
When there is not GNU tool-chain in CLion available the preset is not listed and cannot be selected.
In the current CMakePresets.json file uses tool-chains GNU , GW, GA, MinGW and MSVC.
Build presets are also prevented from builds and listing from the command-line with a condition section.

{
  "version": 6,
  "cmakeMinimumRequired": {
    "major": 3,
    "minor": 25,
    "patch": 0
  },
  "configurePresets": [
    {
      "name": ".tc-gnu",
      "description": "Toolchain for generic GNU",
      "hidden": true,
      "toolchainFile": "${sourceDir}/libpiper/cmake/toolchain-gnu.cmake",
      "cacheVariables": {
        "SF_COMPILER": {
          "type": "STRING",
          "value": "gnu"
        }
      },
      "vendor": {
        "jetbrains.com/clion": {
          "toolchain": "GNU"
        }
      }
    },
  ],
  "buildPresets": [
    {
      "name": ".build",
      "hidden": true,
      "cleanFirst": false,
      "verbose": false
    },
    {
      "name": ".lnx-only",
      "hidden": true,
      "condition": {
        "type": "equals",
        "lhs": "${hostSystemName}",
        "rhs": "Linux"
      }
    }
  ]
}

Build Automation

The build.py script with build.ini and CMakePresets.json enables building across:

  • 5 toolchains (msvc, gnu, mingw, gw, ga)
  • 4 platform modes (native, wine, docker, docker+wine)
  • 11 total combinations with proper environment isolation

On the YouTube channel the workings of the Python script is shown for Linux and Windows.
The build.py is originally from my repository sf-cmake.

Below a list of configured toolchain and platform combinations where the Docker image (Ubuntu 24.04 baed)
has all toolchains on board (except mingw since it is superfluous since the gw cross compiler).
Docker needs running the image with special options to allow using fuse-zip (mounting zip-files).

  1. msvc (native Windows)
  2. msvc.wine (MSVC in Wine on Linux)
  3. msvc.wine.docker (MSVC in Wine inside Docker)
  4. gnu (native Linux GCC x86_64 or aarch64)
  5. gnu.docker (GCC in Docker)
  6. mingw (native Windows MinGW)
  7. mingw.wine (MinGW in Wine on Linux)
  8. gw (cross-compile Windows on Linux)
  9. gw.docker (cross-compile Windows in Docker)
  10. ga (native Linux GCC alternate)
  11. ga.docker (GCC alternate in Docker)

Note: Windows contains an older version of the onnxruntime.dll which is part of the OS.
Directing the PATH to the location in the project is not working since the Windows\system32 directory
has a higher priority so the DLL must be copied into the executable directory. :(
Wine does not have this problem :)

Arjan van Olphen and others added 16 commits December 31, 2025 15:27
…enforce C++ styling consistency.

Added libpiper directory as subdirectory to the main CMakeLists.txt which allows using this repo a 'FetchContent' compatible library.
Added CMakePresets.json to configure cross-compilers presets using predefined toolchain cmake files in the cmake directory like 'cmake/toolchain-gw.cmake'.
Added Windows target hack in 'piper.cpp' to allow the static 'espeak-ng' library to link successfully since the Windows target is forced to be shared.
Added CLI app to allow an integrated C++ test.
Fix for a declaration of global instance in a header which only MSVC bugs on.
Fixes to build the CLI app from Windows with MinGW and MSVC and also from Linux/Wine.
Added file 'CMakePresets.json' to configuring separate directories and environments for the different toolchains builds. Added an integration test in cmake 'piper-cli' application.
Added generic build.py script with build.ini configuration to build the targets with all toolchains on Linux using Wine or Docker + Wine or native on Windows.
…ays and removed 'clang-format' from using remote apt repo.
Update ZIP extraction in build.py to properly restore Unix file permissions
when extracting archives on Unix-like systems. The fix correctly applies
the external_attr permissions while safely handling Windows platforms and
filesystem errors.
Added `get_github_release` and `extract_by_url` functions to streamline fetching and extracting GitHub release assets. Improved toolchain installation support, including a new "doxygen" installation option for Linux systems. Simplified WINEPATH handling by centralizing logic.
Extend `build.py` to include "qemu" as an option for the `--required` argument. Add logic to install QEMU-related packages for running alternate architectures in a Docker container. Update documentation in the help menu for clarity.
Added `get_github_release` and `extract_by_url` functions to streamline fetching and extracting GitHub release assets. Improved toolchain installation support, including a new "doxygen" installation option for Linux systems. Simplified WINEPATH handling by centralizing logic.
…PIPER_BUILD_LIB_ONLY`, add scale factor argument for CLI, enhance error handling, and update `espeak-ng-data` handling for submodules.
…lags to conditionally handle intonation files and tests for subproject builds.
Extend `build.ini` and `build.py` to include a new `env.mingw.wine.docker` environment for running MinGW compiler in Docker and Wine. Adjust path and suffix configurations accordingly.
…dling

Add new hidden toolchain configurations (`.tc-gnu`, `.tc-ga`, `.tc-gw`, `.tc-mingw`) in `CMakePresets.json`. Refactor presets to inherit from these toolchains for consistency. Update `build.py` to handle toolchain-specific environment variables (`LD_LIBRARY_PATH`, `SF_EXEC_DIR_SUFFIX`, etc.) and add recursive removal of `CMakeCache.txt` for fresh builds. Improve error messages, handling for environment configuration, and preset inheritance validation logic.
Differentiate between Windows build and others where the dynamic onnxruntime dynlibs are to be copied to.
Due to Docker v29.4.2+ new security restrictions, sockets cannot be opened without an additional security option added to the command line. Updating the build.py from the original.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant