sdist doesn't package source_dir

[njzjz]

Per Python documentation

The following files are included in a source distribution by default:

all C source files mentioned in the ext_modules or libraries setup() arguments

However, when I use cmake-build-extension, I have to manually add the source into sdist.

[njzjz]

The workaround is to add the source directory to MANIFEST.in:

recursive-include source *

[diegoferigo]

Hi @njzjz, thanks for reporting. Can you please provide more information about your setting? I've just tried the following two cases, and the sdist archive always contains the entire content of the git repo.

Test 1: the example of this repository

~ 🅒 /conda  
❯ cd /tmp

/tmp 🅒 /conda  
❯ git clone https://github.com/diegoferigo/cmake-build-extension
Cloning into 'cmake-build-extension'...
remote: Enumerating objects: 573, done.
remote: Counting objects: 100% (323/323), done.
remote: Compressing objects: 100% (160/160), done.
remote: Total 573 (delta 192), reused 244 (delta 133), pack-reused 250
Receiving objects: 100% (573/573), 157.23 KiB | 1000.00 KiB/s, done.
Resolving deltas: 100% (263/263), done.

/tmp 🅒 /conda  
❯ cd cmake-build-extension/example

cmake-build-extension/example on  master via △ v3.24.1 via 🐍 v3.8.13 🅒 /conda  
❯ python -m build .
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (cmake_build_extension, numpy, pybind11, setuptools, setuptools_scm[toml], wheel)
* Getting dependencies for sdist...
running egg_info
creating src/mymath.egg-info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
running check
creating mymath-0.0.0
creating mymath-0.0.0/bindings_pybind11
creating mymath-0.0.0/bindings_swig
creating mymath-0.0.0/src
creating mymath-0.0.0/src/mymath.egg-info
creating mymath-0.0.0/tests
copying files to mymath-0.0.0...
copying .clang-format -> mymath-0.0.0
copying CMakeLists.txt -> mymath-0.0.0
copying README.md -> mymath-0.0.0
copying pyproject.toml -> mymath-0.0.0
copying setup.cfg -> mymath-0.0.0
copying setup.py -> mymath-0.0.0
copying bindings_pybind11/CMakeLists.txt -> mymath-0.0.0/bindings_pybind11
copying bindings_pybind11/bindings.cpp -> mymath-0.0.0/bindings_pybind11
copying bindings_swig/CMakeLists.txt -> mymath-0.0.0/bindings_swig
copying bindings_swig/bindings.i -> mymath-0.0.0/bindings_swig
copying bindings_swig/numpy.i -> mymath-0.0.0/bindings_swig
copying src/mymath.cpp -> mymath-0.0.0/src
copying src/mymath.h -> mymath-0.0.0/src
copying src/print_answer.cpp -> mymath-0.0.0/src
copying src/mymath.egg-info/PKG-INFO -> mymath-0.0.0/src/mymath.egg-info
copying src/mymath.egg-info/SOURCES.txt -> mymath-0.0.0/src/mymath.egg-info
copying src/mymath.egg-info/dependency_links.txt -> mymath-0.0.0/src/mymath.egg-info
copying src/mymath.egg-info/entry_points.txt -> mymath-0.0.0/src/mymath.egg-info
copying src/mymath.egg-info/not-zip-safe -> mymath-0.0.0/src/mymath.egg-info
copying src/mymath.egg-info/requires.txt -> mymath-0.0.0/src/mymath.egg-info
copying src/mymath.egg-info/top_level.txt -> mymath-0.0.0/src/mymath.egg-info
copying tests/test_pybind11.py -> mymath-0.0.0/tests
copying tests/test_swig.py -> mymath-0.0.0/tests
Writing mymath-0.0.0/setup.cfg
Creating tar archive
removing 'mymath-0.0.0' (and everything under it)
* Building wheel from sdist
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (cmake_build_extension, numpy, pybind11, setuptools, setuptools_scm[toml], wheel)
* Getting dependencies for wheel...
running egg_info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
listing git files failed - pretending there aren't any
reading manifest file 'src/mymath.egg-info/SOURCES.txt'
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
running bdist_wheel
running build
running build_ext

==> Configuring:
$ cmake -S /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0 -B /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_SwigBindings -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig -DCMAKE_MAKE_PROGRAM=/tmp/build-env-blgp3zxe/bin/ninja -DPython3_ROOT_DIR=/tmp/build-env-blgp3zxe -DCALL_FROM_SETUP_PY:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=OFF -DEXAMPLE_WITH_SWIG:BOOL=ON -DEXAMPLE_WITH_PYBIND11:BOOL=OFF

==> Building:
$ cmake --build /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_SwigBindings --config Release

==> Installing:
$ cmake --install /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_SwigBindings

-- The C compiler identification is GNU 10.4.0
-- The CXX compiler identification is GNU 10.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /conda/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /conda/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /tmp/build-env-blgp3zxe/bin/python3.8 (found version "3.8.13") found components: Interpreter Development.Module NumPy 
-- Found SWIG: /conda/bin/swig (found suitable version "4.0.2", minimum required is "4.0")  
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_SwigBindings
[7/7] Linking CXX shared module bindings_swig/mymath/_swig_bindings.so
-- Install configuration: "Release"
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/lib/libmymath.a
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/include/mymath.h
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/bin/print_answer_swig
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/bindings.py
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_swig/_swig_bindings.so

==> Configuring:
$ cmake -S /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0 -B /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_Pybind11Bindings -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11 -DCMAKE_MAKE_PROGRAM=/tmp/build-env-blgp3zxe/bin/ninja -DPython3_ROOT_DIR=/tmp/build-env-blgp3zxe -DCALL_FROM_SETUP_PY:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=OFF -DEXAMPLE_WITH_SWIG:BOOL=OFF -DEXAMPLE_WITH_PYBIND11:BOOL=ON

==> Building:
$ cmake --build /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_Pybind11Bindings --config Release

==> Installing:
$ cmake --install /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_Pybind11Bindings

-- The C compiler identification is GNU 10.4.0
-- The CXX compiler identification is GNU 10.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /conda/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /conda/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /tmp/build-env-blgp3zxe/bin/python3.8 (found version "3.8.13") found components: Interpreter Development.Module NumPy 
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /tmp/build-env-blgp3zxe/lib/python3.8/site-packages/pybind11/include (found version "2.10.0")
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/temp.linux-x86_64-cpython-38_Pybind11Bindings
[6/6] Linking CXX shared module bindings.../bindings.cpython-38-x86_64-linux-gnu.so
-- Install configuration: "Release"
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/libmymath.a
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/include/mymath.h
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/bin/print_answer_pybind11
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathTargets.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathTargets-release.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathConfig.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathConfigVersion.cmake
-- Installing: /tmp/build-via-sdist-sjr8y5gm/mymath-0.0.0/build/lib.linux-x86_64-cpython-38/mymath_pybind11/bindings.cpython-38-x86_64-linux-gnu.so
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/mymath_pybind11
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/lib
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/libmymath.a -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathConfig.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathTargets.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathTargets-release.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/lib/cmake/MyMath/MyMathConfigVersion.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/bindings.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/mymath_pybind11
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/bin
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/bin/__main__.py -> build/bdist.linux-x86_64/wheel/mymath_pybind11/bin
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/bin/print_answer_pybind11 -> build/bdist.linux-x86_64/wheel/mymath_pybind11/bin
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/include
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/include/mymath.h -> build/bdist.linux-x86_64/wheel/mymath_pybind11/include
copying build/lib.linux-x86_64-cpython-38/mymath_pybind11/__init__.py -> build/bdist.linux-x86_64/wheel/mymath_pybind11
creating build/bdist.linux-x86_64/wheel/mymath_swig
creating build/bdist.linux-x86_64/wheel/mymath_swig/lib
copying build/lib.linux-x86_64-cpython-38/mymath_swig/lib/libmymath.a -> build/bdist.linux-x86_64/wheel/mymath_swig/lib
creating build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake
creating build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-cpython-38/mymath_swig/bindings.py -> build/bdist.linux-x86_64/wheel/mymath_swig
copying build/lib.linux-x86_64-cpython-38/mymath_swig/_swig_bindings.so -> build/bdist.linux-x86_64/wheel/mymath_swig
creating build/bdist.linux-x86_64/wheel/mymath_swig/bin
copying build/lib.linux-x86_64-cpython-38/mymath_swig/bin/__main__.py -> build/bdist.linux-x86_64/wheel/mymath_swig/bin
copying build/lib.linux-x86_64-cpython-38/mymath_swig/bin/print_answer_swig -> build/bdist.linux-x86_64/wheel/mymath_swig/bin
creating build/bdist.linux-x86_64/wheel/mymath_swig/include
copying build/lib.linux-x86_64-cpython-38/mymath_swig/include/mymath.h -> build/bdist.linux-x86_64/wheel/mymath_swig/include
copying build/lib.linux-x86_64-cpython-38/mymath_swig/__init__.py -> build/bdist.linux-x86_64/wheel/mymath_swig
running install_egg_info
running egg_info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
listing git files failed - pretending there aren't any
reading manifest file 'src/mymath.egg-info/SOURCES.txt'
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
Copying src/mymath.egg-info to build/bdist.linux-x86_64/wheel/mymath-0.0.0-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/mymath-0.0.0.dist-info/WHEEL
creating '/tmp/cmake-build-extension/example/dist/tmpb4pl09_u/mymath-0.0.0-cp38-cp38-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'mymath_pybind11/__init__.py'
adding 'mymath_pybind11/bindings.cpython-38-x86_64-linux-gnu.so'
adding 'mymath_pybind11/bin/__main__.py'
adding 'mymath_pybind11/bin/print_answer_pybind11'
adding 'mymath_pybind11/include/mymath.h'
adding 'mymath_pybind11/lib/libmymath.a'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathConfig.cmake'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathConfigVersion.cmake'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathTargets-release.cmake'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathTargets.cmake'
adding 'mymath_swig/__init__.py'
adding 'mymath_swig/_swig_bindings.so'
adding 'mymath_swig/bindings.py'
adding 'mymath_swig/bin/__main__.py'
adding 'mymath_swig/bin/print_answer_swig'
adding 'mymath_swig/include/mymath.h'
adding 'mymath_swig/lib/libmymath.a'
adding 'mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake'
adding 'mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake'
adding 'mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake'
adding 'mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake'
adding 'mymath-0.0.0.dist-info/METADATA'
adding 'mymath-0.0.0.dist-info/WHEEL'
adding 'mymath-0.0.0.dist-info/entry_points.txt'
adding 'mymath-0.0.0.dist-info/top_level.txt'
adding 'mymath-0.0.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built mymath-0.0.0.tar.gz and mymath-0.0.0-cp38-cp38-linux_x86_64.whl

cmake-build-extension/example on  master via △ v3.24.1 via 🐍 v3.8.13 🅒 /conda  took 39s 
❯ cd dist/

cmake-build-extension/example/dist on  master 🅒 /conda  
❯ ls
mymath-0.0.0-cp38-cp38-linux_x86_64.whl  mymath-0.0.0.tar.gz

cmake-build-extension/example/dist on  master 🅒 /conda  
❯ tar xfv mymath-0.0.0.tar.gz
mymath-0.0.0/
mymath-0.0.0/.clang-format
mymath-0.0.0/CMakeLists.txt
mymath-0.0.0/PKG-INFO
mymath-0.0.0/README.md
mymath-0.0.0/bindings_pybind11/
mymath-0.0.0/bindings_pybind11/CMakeLists.txt
mymath-0.0.0/bindings_pybind11/bindings.cpp
mymath-0.0.0/bindings_swig/
mymath-0.0.0/bindings_swig/CMakeLists.txt
mymath-0.0.0/bindings_swig/bindings.i
mymath-0.0.0/bindings_swig/numpy.i
mymath-0.0.0/pyproject.toml
mymath-0.0.0/setup.cfg
mymath-0.0.0/setup.py
mymath-0.0.0/src/
mymath-0.0.0/src/mymath.cpp
mymath-0.0.0/src/mymath.egg-info/
mymath-0.0.0/src/mymath.egg-info/PKG-INFO
mymath-0.0.0/src/mymath.egg-info/SOURCES.txt
mymath-0.0.0/src/mymath.egg-info/dependency_links.txt
mymath-0.0.0/src/mymath.egg-info/entry_points.txt
mymath-0.0.0/src/mymath.egg-info/not-zip-safe
mymath-0.0.0/src/mymath.egg-info/requires.txt
mymath-0.0.0/src/mymath.egg-info/top_level.txt
mymath-0.0.0/src/mymath.h
mymath-0.0.0/src/print_answer.cpp
mymath-0.0.0/tests/
mymath-0.0.0/tests/test_pybind11.py
mymath-0.0.0/tests/test_swig.py

I can see bindings.cpp, mymath.h, print_answer.cpp as part of the sdist archive.

Test 2: gazebo-yarp-synchronizer

This is a rather old package but it's simple enough to be considered another didactic example.

~ 🅒 /conda  
❯ cd /tmp

/tmp 🅒 /conda  
❯ git clone https://github.com/diegoferigo/gazebo-yarp-synchronizer
Cloning into 'gazebo-yarp-synchronizer'...
remote: Enumerating objects: 221, done.
remote: Counting objects: 100% (221/221), done.
remote: Compressing objects: 100% (174/174), done.
remote: Total 221 (delta 104), reused 131 (delta 23), pack-reused 0
Receiving objects: 100% (221/221), 62.81 KiB | 1.05 MiB/s, done.
Resolving deltas: 100% (104/104), done.

/tmp 🅒 /conda  
❯ cd gazebo-yarp-synchronizer/

gazebo-yarp-synchronizer on  main via △ v3.24.1 via 🐍 v3.8.13 🅒 /conda  
❯ python -m build .
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (cmake-build-extension, pybind11, setuptools>=45, setuptools_scm[toml]>=6.0, wheel, yarp-middleware, ycm-cmake-modules)
* Getting dependencies for sdist...
running egg_info
creating gazebo_yarp_synchronizer.egg-info
writing gazebo_yarp_synchronizer.egg-info/PKG-INFO
writing dependency_links to gazebo_yarp_synchronizer.egg-info/dependency_links.txt
writing top-level names to gazebo_yarp_synchronizer.egg-info/top_level.txt
writing manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing gazebo_yarp_synchronizer.egg-info/PKG-INFO
writing dependency_links to gazebo_yarp_synchronizer.egg-info/dependency_links.txt
writing top-level names to gazebo_yarp_synchronizer.egg-info/top_level.txt
adding license file 'LICENSE'
writing manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
running check
creating gazebo_yarp_synchronizer-0.9.1
creating gazebo_yarp_synchronizer-0.9.1/.github
creating gazebo_yarp_synchronizer-0.9.1/.github/workflows
creating gazebo_yarp_synchronizer-0.9.1/cmake
creating gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info
creating gazebo_yarp_synchronizer-0.9.1/src
creating gazebo_yarp_synchronizer-0.9.1/src/bindings
creating gazebo_yarp_synchronizer-0.9.1/src/gazebo
creating gazebo_yarp_synchronizer-0.9.1/src/gazebo/include
creating gazebo_yarp_synchronizer-0.9.1/src/gazebo/include/gys
creating gazebo_yarp_synchronizer-0.9.1/src/thrift
copying files to gazebo_yarp_synchronizer-0.9.1...
copying .clang-format -> gazebo_yarp_synchronizer-0.9.1
copying .gitignore -> gazebo_yarp_synchronizer-0.9.1
copying CMakeLists.txt -> gazebo_yarp_synchronizer-0.9.1
copying LICENSE -> gazebo_yarp_synchronizer-0.9.1
copying README.md -> gazebo_yarp_synchronizer-0.9.1
copying pyproject.toml -> gazebo_yarp_synchronizer-0.9.1
copying setup.cfg -> gazebo_yarp_synchronizer-0.9.1
copying setup.py -> gazebo_yarp_synchronizer-0.9.1
copying .github/workflows/python.yml -> gazebo_yarp_synchronizer-0.9.1/.github/workflows
copying cmake/AddUninstallTarget.cmake -> gazebo_yarp_synchronizer-0.9.1/cmake
copying cmake/InstallBasicPackageFiles.cmake -> gazebo_yarp_synchronizer-0.9.1/cmake
copying gazebo_yarp_synchronizer.egg-info/PKG-INFO -> gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info
copying gazebo_yarp_synchronizer.egg-info/SOURCES.txt -> gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info
copying gazebo_yarp_synchronizer.egg-info/dependency_links.txt -> gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info
copying gazebo_yarp_synchronizer.egg-info/not-zip-safe -> gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info
copying gazebo_yarp_synchronizer.egg-info/top_level.txt -> gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info
copying src/CMakeLists.txt -> gazebo_yarp_synchronizer-0.9.1/src
copying src/bindings/CMakeLists.txt -> gazebo_yarp_synchronizer-0.9.1/src/bindings
copying src/bindings/gazebo_yarp_synchronizer.cpp -> gazebo_yarp_synchronizer-0.9.1/src/bindings
copying src/gazebo/CMakeLists.txt -> gazebo_yarp_synchronizer-0.9.1/src/gazebo
copying src/gazebo/GazeboYarpSynchronizer.cpp -> gazebo_yarp_synchronizer-0.9.1/src/gazebo
copying src/gazebo/include/gys/GazeboYarpSynchronizer.h -> gazebo_yarp_synchronizer-0.9.1/src/gazebo/include/gys
copying src/thrift/CMakeLists.txt -> gazebo_yarp_synchronizer-0.9.1/src/thrift
copying src/thrift/clock_rpc.thrift -> gazebo_yarp_synchronizer-0.9.1/src/thrift
Writing gazebo_yarp_synchronizer-0.9.1/setup.cfg
Creating tar archive
removing 'gazebo_yarp_synchronizer-0.9.1' (and everything under it)
* Building wheel from sdist
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (cmake-build-extension, pybind11, setuptools>=45, setuptools_scm[toml]>=6.0, wheel, yarp-middleware, ycm-cmake-modules)
* Getting dependencies for wheel...
running egg_info
writing gazebo_yarp_synchronizer.egg-info/PKG-INFO
writing dependency_links to gazebo_yarp_synchronizer.egg-info/dependency_links.txt
writing top-level names to gazebo_yarp_synchronizer.egg-info/top_level.txt
listing git files failed - pretending there aren't any
reading manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
running bdist_wheel
running build
running build_ext

==> Configuring:
$ cmake -S /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1 -B /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/temp.linux-x86_64-cpython-38_InstallAllTargets -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer -DCMAKE_MAKE_PROGRAM=/tmp/build-env-h_ylfu2o/bin/ninja -DPython3_EXECUTABLE:PATH=/tmp/build-env-h_ylfu2o/bin/python -DBUILD_SHARED_LIBS:BOOL=OFF -DGYS_CALL_FROM_SETUP_PY:BOOL=ON

==> Building:
$ cmake --build /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/temp.linux-x86_64-cpython-38_InstallAllTargets --config Release

==> Installing:
$ cmake --install /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/temp.linux-x86_64-cpython-38_InstallAllTargets

-- The CXX compiler identification is GNU 10.4.0
-- The C compiler identification is GNU 10.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /conda/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /conda/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found YARP: /usr/local/src/robotology-superbuild/build/install/lib/cmake/YARP (found version "3.7.2+9-20220908.4+gitb6b13ec5c")
-- Found YARP: /usr/local/src/robotology-superbuild/build/install/lib/cmake/YARP (found version "3.7.2+9-20220908.4+gitb6b13ec5c")
-- Found Python3: /tmp/build-env-h_ylfu2o/bin/python (found version "3.8.13") found components: Interpreter Development.Module 
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /tmp/build-env-h_ylfu2o/lib/python3.8/site-packages/pybind11/include (found version "2.10.0")
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/temp.linux-x86_64-cpython-38_InstallAllTargets
[7/7] Linking CXX shared module gazebo_yarp_synchronizer/bindings.cpython-38-x86_64-linux-gnu.so
-- Install configuration: "Release"
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/libClockRpc.a
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/libGazeboYarpSynchronizer.a
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/include/gys/GazeboYarpSynchronizer.h
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/__init__.py
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/bindings.cpython-38-x86_64-linux-gnu.so
-- Set runtime path of "/tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/bindings.cpython-38-x86_64-linux-gnu.so" to ""
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerConfigVersion.cmake
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerConfig.cmake
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerTargets.cmake
-- Installing: /tmp/build-via-sdist-qo_c1o61/gazebo_yarp_synchronizer-0.9.1/build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerTargets-release.cmake
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/libGazeboYarpSynchronizer.a -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib/cmake
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerTargets.cmake -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerConfig.cmake -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerConfigVersion.cmake -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerTargets-release.cmake -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/lib/libClockRpc.a -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/lib
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/bindings.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/include
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/include/gys
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/include/gys/GazeboYarpSynchronizer.h -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer/include/gys
copying build/lib.linux-x86_64-cpython-38/gazebo_yarp_synchronizer/__init__.py -> build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer
running install_egg_info
running egg_info
writing gazebo_yarp_synchronizer.egg-info/PKG-INFO
writing dependency_links to gazebo_yarp_synchronizer.egg-info/dependency_links.txt
writing top-level names to gazebo_yarp_synchronizer.egg-info/top_level.txt
listing git files failed - pretending there aren't any
reading manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'gazebo_yarp_synchronizer.egg-info/SOURCES.txt'
Copying gazebo_yarp_synchronizer.egg-info to build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer-0.9.1-py3.8.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/gazebo_yarp_synchronizer-0.9.1.dist-info/WHEEL
creating '/tmp/gazebo-yarp-synchronizer/dist/tmpymcdvn2k/gazebo_yarp_synchronizer-0.9.1-cp38-cp38-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'gazebo_yarp_synchronizer/__init__.py'
adding 'gazebo_yarp_synchronizer/bindings.cpython-38-x86_64-linux-gnu.so'
adding 'gazebo_yarp_synchronizer/include/gys/GazeboYarpSynchronizer.h'
adding 'gazebo_yarp_synchronizer/lib/libClockRpc.a'
adding 'gazebo_yarp_synchronizer/lib/libGazeboYarpSynchronizer.a'
adding 'gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerConfig.cmake'
adding 'gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerConfigVersion.cmake'
adding 'gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerTargets-release.cmake'
adding 'gazebo_yarp_synchronizer/lib/cmake/GazeboYarpSynchronizer/GazeboYarpSynchronizerTargets.cmake'
adding 'gazebo_yarp_synchronizer-0.9.1.dist-info/LICENSE'
adding 'gazebo_yarp_synchronizer-0.9.1.dist-info/METADATA'
adding 'gazebo_yarp_synchronizer-0.9.1.dist-info/WHEEL'
adding 'gazebo_yarp_synchronizer-0.9.1.dist-info/top_level.txt'
adding 'gazebo_yarp_synchronizer-0.9.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built gazebo_yarp_synchronizer-0.9.1.tar.gz and gazebo_yarp_synchronizer-0.9.1-cp38-cp38-linux_x86_64.whl

gazebo-yarp-synchronizer on  main via △ v3.24.1 via 🐍 v3.8.13 🅒 /conda  took 27s 
❯ cd dist/

gazebo-yarp-synchronizer/dist on  main 🅒 /conda  
❯ tar xvf gazebo_yarp_synchronizer-0.9.1.tar.gz 
gazebo_yarp_synchronizer-0.9.1/
gazebo_yarp_synchronizer-0.9.1/.clang-format
gazebo_yarp_synchronizer-0.9.1/.github/
gazebo_yarp_synchronizer-0.9.1/.github/workflows/
gazebo_yarp_synchronizer-0.9.1/.github/workflows/python.yml
gazebo_yarp_synchronizer-0.9.1/.gitignore
gazebo_yarp_synchronizer-0.9.1/CMakeLists.txt
gazebo_yarp_synchronizer-0.9.1/LICENSE
gazebo_yarp_synchronizer-0.9.1/PKG-INFO
gazebo_yarp_synchronizer-0.9.1/README.md
gazebo_yarp_synchronizer-0.9.1/cmake/
gazebo_yarp_synchronizer-0.9.1/cmake/AddUninstallTarget.cmake
gazebo_yarp_synchronizer-0.9.1/cmake/InstallBasicPackageFiles.cmake
gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info/
gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info/PKG-INFO
gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info/SOURCES.txt
gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info/dependency_links.txt
gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info/not-zip-safe
gazebo_yarp_synchronizer-0.9.1/gazebo_yarp_synchronizer.egg-info/top_level.txt
gazebo_yarp_synchronizer-0.9.1/pyproject.toml
gazebo_yarp_synchronizer-0.9.1/setup.cfg
gazebo_yarp_synchronizer-0.9.1/setup.py
gazebo_yarp_synchronizer-0.9.1/src/
gazebo_yarp_synchronizer-0.9.1/src/CMakeLists.txt
gazebo_yarp_synchronizer-0.9.1/src/bindings/
gazebo_yarp_synchronizer-0.9.1/src/bindings/CMakeLists.txt
gazebo_yarp_synchronizer-0.9.1/src/bindings/gazebo_yarp_synchronizer.cpp
gazebo_yarp_synchronizer-0.9.1/src/gazebo/
gazebo_yarp_synchronizer-0.9.1/src/gazebo/CMakeLists.txt
gazebo_yarp_synchronizer-0.9.1/src/gazebo/GazeboYarpSynchronizer.cpp
gazebo_yarp_synchronizer-0.9.1/src/gazebo/include/
gazebo_yarp_synchronizer-0.9.1/src/gazebo/include/gys/
gazebo_yarp_synchronizer-0.9.1/src/gazebo/include/gys/GazeboYarpSynchronizer.h
gazebo_yarp_synchronizer-0.9.1/src/thrift/
gazebo_yarp_synchronizer-0.9.1/src/thrift/CMakeLists.txt
gazebo_yarp_synchronizer-0.9.1/src/thrift/clock_rpc.thrift

Also in this case, I can see the entire content of the git repository bundled inside the sdist archive.

[njzjz]

Hi, my repo is https://github.com/njzjz/openbabel-wheel.

The build without MANIFEST.in:
https://github.com/njzjz/openbabel-wheel/actions/runs/3289094388/jobs/5420216496

The build with MANIFEST.in:
https://github.com/njzjz/openbabel-wheel/actions/runs/3293366858/jobs/5429793125

[diegoferigo]

At first look, it seems that your problem origins from the fact that source in your case is a git submodule and not a real folder. I'm not sure the problem is cmake-build-extension, I'd recommend to first check if git submodules are supported in the first place by setuptools_scm that is responsible to populate the list of files that are packed into the sdist archive.

[diegoferigo]

Actually, you might be lucky enough that our cmake_build_extension.GitSdistFolder custom sdist command could solve your problem. Apply the following patch and try to build the sdist from a repository that has the submodules all initialized (i.e. after calling git submodule update --init --recursive):

diff --git a/setup.py b/setup.py
index 89aefb9..ddb5484 100644
--- a/setup.py
+++ b/setup.py
@@ -3,6 +3,7 @@ import os
 import sys
 from pathlib import Path
 
+import cmake_build_extension
 from cmake_build_extension import BuildExtension, CMakeExtension
 from setuptools import setup
 from distutils import sysconfig
@@ -30,5 +31,5 @@ setup(
                             *cmake_options,
                        ]),
     ],
-    cmdclass=dict(build_ext=BuildExtension),
+    cmdclass=dict(build_ext=BuildExtension, sdist=cmake_build_extension.GitSdistFolder),
 )

Note that your package will explicitly depend on cmake-build-extension and there's a new issue regarding potential unwanted effect in downstream libraries. I suggest to follow #40 since there could be changes in next releases.

[Roy-Kid]

I got the same issue too. I can run the example, but when I setup a test project, the extension can not include the cpp-related code in to sdist(only the .py). I would like to know where to control the extension's behavior to determine what file is included in the sdist. If I don't use the gitsdistfolder, how can I choose the included files? by manifest?

[Roy-Kid]

I have many tests on the example. For instance, I rename the bindings_pybind11 with cpp and change related code, and then the setuptools can not find the cmakelists and cpp and the code under it. Could you please tell me how to control files in sdist?