.. _devel-cgal-swig-bindings-packaging: ============================ cgal-swig-bindings packaging ============================ Purpose ------- The ``cgal-swig-bindings`` package contains the prebuilt Python bindings used by Code_TYMPAN to access CGAL primitives from Python code. Building these bindings from source is time-consuming on Windows, because SWIG generates the Python bindings and the generated C++ code is then compiled against CGAL. For this reason, Code_TYMPAN publishes prebuilt ``cgal-swig-bindings`` packages in the GitLab Package Registry using Conan 2. This page explains how to create and publish a new package version. This task is only required when one of the following components changes: * CGAL version; * Python version; * SWIG version; * Windows compiler/toolset or ABI; * generated binding contents. Package naming convention ------------------------- The package name is: .. code-block:: text cgal-swig-bindings The package version must explicitly reflect the main ABI-sensitive components: .. code-block:: text ---cgal-py-swig For example: .. code-block:: text cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable This means: .. code-block:: text Architecture: x86_64 Platform: Windows Compiler: MSVC v143 CGAL: 5.6.1 Python: 3.13 SWIG: 4.0.2 When upgrading from Python 3.13 to Python 3.14, the package reference should therefore become for example: .. code-block:: text cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py314-swig402@tympan/stable When upgrading from CGAL 5.6.1 to CGAL 6.0, the package reference should become for example: .. code-block:: text cgal-swig-bindings/x86_64-windows-msvc143-cgal60-py313-swig402@tympan/stable .. warning:: The Python version is part of the package version because compiled Python extension modules are tied to the Python ABI. A package built for Python 3.13 must not be reused with another Python version. Published package versions -------------------------- .. list-table:: :header-rows: 1 * - Package reference - Architecture - Platform - Compiler - Python - CGAL - SWIG - Status * - ``cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable`` - x86_64 - Windows - MSVC v143 - 3.13 - 5.6.1 - 4.0.2 - Current package Prerequisites ------------- The following tools and dependencies must be available on the packaging machine: .. code-block:: text Windows x86_64 Visual Studio 2022 / MSVC v143 Python 3.13 Conan 2 CGAL 5.6.1 SWIG 4.0.2 Code_TYMPAN sources The Python virtual environment used for Code_TYMPAN development already installs Conan through ``requirements-dev.txt``. Activate the virtual environment before running Conan commands: .. code-block:: bat C:\dists\python\venv313tympan\Scripts\activate.bat Check Conan: .. code-block:: bat conan --version If this is the first Conan use on the machine, initialize a profile: .. code-block:: bat conan profile detect Build the bindings from source ------------------------------ To create a new package version, first build Code_TYMPAN once with ``cgal-swig-bindings`` built from source. Make sure that the prebuilt bindings option is disabled: .. code-block:: bat -DTYMPAN_USE_PREBUILT_CGAL_BINDINGS=OFF or simply omit the option, since ``OFF`` is the default value. A typical Release build produces: .. code-block:: text C:\projects\code_tympan_install\cython\CGAL C:\projects\code_tympan_build\3rdparty\Install\CGAL_bindings\bin The first directory contains the Python ``CGAL`` package. The second directory contains the runtime DLLs generated by the ``cgal-swig-bindings`` build. Prepare the Conan recipe directory ---------------------------------- The Conan recipe used to package ``cgal-swig-bindings`` is versioned in the Code_TYMPAN repository: .. code-block:: text tools\conan\cgal-swig-bindings\conanfile.py The helper script used to refresh the package payload is also versioned there: .. code-block:: text tools\conan\cgal-swig-bindings\refresh_package_payload.bat The expected layout is: .. code-block:: text tools\conan\cgal-swig-bindings conanfile.py refresh_package_payload.bat .gitignore package_payload CGAL bin The ``package_payload`` directory is generated locally and must not be committed. Refresh the package payload --------------------------- From the Conan recipe directory: .. code-block:: bat cd /d C:\projects\code_tympan\tools\conan\cgal-swig-bindings Refresh the payload with: .. code-block:: bat refresh_package_payload.bat The script uses the following default paths: .. code-block:: text Code_TYMPAN install directory: C:\projects\code_tympan_install Code_TYMPAN build directory: C:\projects\code_tympan_build These paths can be overridden by passing arguments: .. code-block:: bat refresh_package_payload.bat ^ C:\projects\code_tympan_install ^ C:\projects\code_tympan_build The expected payload layout is: .. code-block:: text package_payload CGAL bin Update the Conan recipe version ------------------------------- Open the Conan recipe versioned in the Code_TYMPAN repository: .. code-block:: text C:\projects\code_tympan\tools\conan\cgal-swig-bindings\conanfile.py Update only the ``version`` field according to the naming convention. For example, for the current CGAL 5.6.1, Python 3.13 and SWIG 4.0.2 package: .. code-block:: python version = "5.6.1-py313-swig402" When upgrading Python from 3.13 to 3.14, update it for example to: .. code-block:: python version = "5.6.1-py314-swig402" When upgrading CGAL from 5.6.1 to 6.0, update it for example to: .. code-block:: python version = "6.0-py313-swig402" The rest of the recipe should usually remain unchanged. The ``conanfile.py`` file is versioned with Code_TYMPAN and should be reviewed like any other packaging-related source file. Create the Conan package locally -------------------------------- From the recipe directory: .. code-block:: bat cd /d C:\projects\code_tympan\tools\conan\cgal-swig-bindings Create the local Conan package: .. code-block:: bat conan export-pkg . --user=tympan --channel=stable -pr:a=default Check that the package exists in the local Conan cache: .. code-block:: bat conan list "cgal-swig-bindings/*" The expected package reference should appear, for example: .. code-block:: text cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable Configure the GitLab Conan remote --------------------------------- The Code_TYMPAN GitLab Conan remote is: .. code-block:: text https://gitlab.com/api/v4/projects/18728059/packages/conan Add it to Conan: .. code-block:: bat conan remote add gitlab https://gitlab.com/api/v4/projects/18728059/packages/conan --force Authenticate with a GitLab token having ``write_package_registry`` permission: .. code-block:: bat conan remote login gitlab "" -p "" For example, the username may look like: .. code-block:: text gitlab+deploy-token-XXXXXXXX .. warning:: Do not store the token in a committed file. If a corporate proxy is required, define it before running Conan commands: .. code-block:: bat set HTTP_PROXY=http://proxy.my-company.com:port set HTTPS_PROXY=http://proxy.my-company.com:port Upload the package to GitLab ---------------------------- Replace the package reference below with the new version before uploading. First run a dry run: .. code-block:: bat conan upload cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable -r=gitlab --dry-run If the dry run is correct, upload the package: .. code-block:: bat conan upload cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable -r=gitlab -c After upload, the package should be visible in the GitLab Package Registry. Minimal post-upload check ------------------------- Use a clean Conan home to make sure the package can be downloaded without using the local Conan cache. .. code-block:: bat set CONAN_HOME=C:\dev\conan-test\conan-home-gitlab rmdir /S /Q C:\dev\conan-test\conan-home-gitlab rmdir /S /Q C:\dev\conan-test\deps conan profile detect conan remote add gitlab https://gitlab.com/api/v4/projects/18728059/packages/conan --force conan remote login gitlab "" -p "" Download and deploy the package: .. code-block:: bat conan install --requires=cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable --remote=gitlab --build=never --deployer-package="cgal-swig-bindings/*" --deployer-folder=C:\dev\conan-test\deps --output-folder=C:\dev\conan-test\conan-output Check the deployed layout: .. code-block:: bat dir C:\dev\conan-test\deps\cgal-swig-bindings\CGAL dir C:\dev\conan-test\deps\cgal-swig-bindings\bin The expected layout is: .. code-block:: text C:\dev\conan-test\deps\cgal-swig-bindings\CGAL C:\dev\conan-test\deps\cgal-swig-bindings\bin If this check succeeds, the package is ready to be consumed by developers and by AppVeyor. Update Code_TYMPAN references ----------------------------- After publishing a new package version, update all references to the package version in the project. Typical locations are: * developer documentation; * AppVeyor configuration; * local build scripts, if any. For example, replace: .. code-block:: text cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable with the new package reference. Troubleshooting --------------- Conan cannot connect to GitLab ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Check whether a corporate proxy must be configured: .. code-block:: bat set HTTP_PROXY=http://proxy.my-company.com:port set HTTPS_PROXY=http://proxy.my-company.com:port Permission error while generating Conan files ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Do not run ``conan install`` from a protected directory such as: .. code-block:: text C:\Windows\System32 Run it from a writable directory, or set an explicit output folder: .. code-block:: bat --output-folder=C:\dev\conan-test\conan-output Wrong Python version ~~~~~~~~~~~~~~~~~~~~ If importing the ``CGAL`` Python package fails after deployment, check that the package version matches the Python version used by Code_TYMPAN. For Python 3.13, the package version must contain: .. code-block:: text py313 For Python 3.14, it must contain: .. code-block:: text py314 Missing DLLs ~~~~~~~~~~~~ If the deployed package contains the ``CGAL`` Python package but not the required DLLs, check the payload before running ``conan export-pkg``: .. code-block:: bat dir C:\projects\code_tympan\tools\conan\cgal-swig-bindings\package_payload\bin The DLLs must come from: .. code-block:: text C:\projects\code_tympan_build\3rdparty\Install\CGAL_bindings\bin