Add multi-compiler support and cross-platform build automation#175
Open
Scanframe wants to merge 16 commits into
Open
Add multi-compiler support and cross-platform build automation#175Scanframe wants to merge 16 commits into
Scanframe wants to merge 16 commits into
Conversation
…enforce C++ styling consistency. Added libpiper directory as subdirectory to the main CMakeLists.txt which allows using this repo a 'FetchContent' compatible library. Added CMakePresets.json to configure cross-compilers presets using predefined toolchain cmake files in the cmake directory like 'cmake/toolchain-gw.cmake'. Added Windows target hack in 'piper.cpp' to allow the static 'espeak-ng' library to link successfully since the Windows target is forced to be shared. Added CLI app to allow an integrated C++ test.
Fix for a declaration of global instance in a header which only MSVC bugs on. Fixes to build the CLI app from Windows with MinGW and MSVC and also from Linux/Wine. Added file 'CMakePresets.json' to configuring separate directories and environments for the different toolchains builds. Added an integration test in cmake 'piper-cli' application. Added generic build.py script with build.ini configuration to build the targets with all toolchains on Linux using Wine or Docker + Wine or native on Windows.
…ays and removed 'clang-format' from using remote apt repo.
Update ZIP extraction in build.py to properly restore Unix file permissions when extracting archives on Unix-like systems. The fix correctly applies the external_attr permissions while safely handling Windows platforms and filesystem errors.
Added `get_github_release` and `extract_by_url` functions to streamline fetching and extracting GitHub release assets. Improved toolchain installation support, including a new "doxygen" installation option for Linux systems. Simplified WINEPATH handling by centralizing logic.
Extend `build.py` to include "qemu" as an option for the `--required` argument. Add logic to install QEMU-related packages for running alternate architectures in a Docker container. Update documentation in the help menu for clarity.
Added `get_github_release` and `extract_by_url` functions to streamline fetching and extracting GitHub release assets. Improved toolchain installation support, including a new "doxygen" installation option for Linux systems. Simplified WINEPATH handling by centralizing logic.
…PIPER_BUILD_LIB_ONLY`, add scale factor argument for CLI, enhance error handling, and update `espeak-ng-data` handling for submodules.
…lags to conditionally handle intonation files and tests for subproject builds.
Extend `build.ini` and `build.py` to include a new `env.mingw.wine.docker` environment for running MinGW compiler in Docker and Wine. Adjust path and suffix configurations accordingly.
…dling Add new hidden toolchain configurations (`.tc-gnu`, `.tc-ga`, `.tc-gw`, `.tc-mingw`) in `CMakePresets.json`. Refactor presets to inherit from these toolchains for consistency. Update `build.py` to handle toolchain-specific environment variables (`LD_LIBRARY_PATH`, `SF_EXEC_DIR_SUFFIX`, etc.) and add recursive removal of `CMakeCache.txt` for fresh builds. Improve error messages, handling for environment configuration, and preset inheritance validation logic.
Differentiate between Windows build and others where the dynamic onnxruntime dynlibs are to be copied to.
Due to Docker v29.4.2+ new security restrictions, sockets cannot be opened without an additional security option added to the command line. Updating the build.py from the original.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request adds comprehensive multi-compiler and cross-platform build support to enable building the Piper library and a CLI test application with various toolchains across different platforms.
This allows developers to test builds across all supported toolchains from a single Linux machine native or via Wine, Docker, Docker+Wine and when needed on Windows.
Minor changes have been made to the C++ source fixing some shortcomings for other toolchains and a serious MSVC only bug.
See it in action on YouTube
The repository Piper TTS Qt-Plugin is depending on these PR's changes.
How To
Simply run this in Linux with
dockerinstalled:Build and Test Linux using its host architecture
Build and Test Linux using architecture
aarch64(cross-compiled)Build and Test for Windows using GNU for Windows (cross-compiled)
The test is executed in Wine.
Build and Test for Windows using MSVC-2022 compiler
Compiler and the test are running in Linux/Wine.
The
output.wavis the testresult located thebin/lnx64-<compiler>orbin/win64-<compiler>directory.Toolchains Available for Building
This a list of all available toolchains and how they are referenced in the CMake configuration.:
gnu- GNU GCC Linux x86_64 or aarch64 which depends on the build hostmingw- MinGW-w64 (GNU GCC for Windows x86_64)gw- GNU GCC cross-compiler for Windows on Linux x86_64ga- Aarch64 GNU GCC cross-compiler on Linux x86_64msvc- Microsoft Visual C++ 2022 x86_64Platform Support:
Key Changes
CMakePresets.jsonwith toolchain-specific configurations and separate build directoriesbuild.pyautomation script withbuild.iniconfiguration supporting all 11 tool/platform combinationsrun-executable.cmakehelper script for cross-platform test executionusing the environment and variables set in the
CMakePresets.jsonfile.piper-cliapplication with CTest.CMakeLists.txtwithCMakePresets.jsonsuitable for working with JetBrains/CLion or VSCode.CLion Autmatic Toolchain Selection
To have the correct toolchain selected in JetBrain's CLion the
vendorsection is used.When there is not
GNUtool-chain in CLion available the preset is not listed and cannot be selected.In the current
CMakePresets.jsonfile uses tool-chainsGNU,GW,GA, MinGW andMSVC.Build presets are also prevented from builds and listing from the command-line with a
conditionsection.{ "version": 6, "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, "configurePresets": [ { "name": ".tc-gnu", "description": "Toolchain for generic GNU", "hidden": true, "toolchainFile": "${sourceDir}/libpiper/cmake/toolchain-gnu.cmake", "cacheVariables": { "SF_COMPILER": { "type": "STRING", "value": "gnu" } }, "vendor": { "jetbrains.com/clion": { "toolchain": "GNU" } } }, ], "buildPresets": [ { "name": ".build", "hidden": true, "cleanFirst": false, "verbose": false }, { "name": ".lnx-only", "hidden": true, "condition": { "type": "equals", "lhs": "${hostSystemName}", "rhs": "Linux" } } ] }Build Automation
The
build.pyscript withbuild.iniandCMakePresets.jsonenables building across:Below a list of configured toolchain and platform combinations where the Docker image (Ubuntu 24.04 baed)
has all toolchains on board (except
mingwsince it is superfluous since thegwcross compiler).Docker needs running the image with special options to allow using fuse-zip (mounting zip-files).