.. _devel-installation:

============
Installation
============

Overview
========

.. _devel-downloading:

Downloading
-----------

| A Git_ repository is available on Gitlab_ at https://gitlab.com/tympan/code_tympan.
| A Gitflow workflow is currently used; so the stable branch is master, and the development branch is develop.
| The latest version of the documentation describe how to build Code_TYMPAN using the branch "develop".

.. _Git: https://git-scm.com/
.. _Gitlab: https://gitlab.com/

The project also requires some third party dependencies (see below for
more details).

Those dependencies are divided into heavy, potentially system-wide,
dependencies and lighter dependencies which can easily be
bundled in a source package along with Code_TYMPAN.

Lightweight dependencies
------------------------

Those dependencies are to be downloaded as a companion source package
and unzipped into the ``3rdparty`` directory. Please refer to the
``3rdparty/README`` file which provides the exact URL for your current
version of Code_TYMPAN.

Major dependencies
------------------

The following dependencies are a bigger deal as their are likely to be
already installed in your development environment in a system-wide
way. If you are running a Linux box, the best way to get them is
through your distribution package manager. If you are running a
Windows box, you should run the dedicated installers.

More details are given in further sections, on an OS and variant basis.

Boost
        Boost is a set of C++ libraries which provide a higher level
        of expressiveness and convenience for C++. Version 1.86 is
        currently used.

Qt 6
        Qt is a toolkit for developing Graphical User Interface and
        providing portable and commonly used facilities such as
        multi-threading or reading XML files... Version 6.8.3
        is currently used.

OpenGL
        is a common 3D rendering API. It is mainly accessed through
        the QtOpenGL components.

CMake
        CMake is a multi platform built tool used to build Code_TYMPAN
        (and CGAL and many other OpenSource projects by the way). A
        version 4.1.1 is currently used.

CGAL
        CGAL is a Computational Geometry library and itself depends
        upon both Boost and Qt and as such needs to be installed in a
        compatible way with your Qt and Boost installation.
        CGAL is used in the Python code for altimetry mesh triangulation and refinement. It is also used
        in the C++ code (business part) for infrastructure polygons triangulation.

Node.js
        Node.js is a free, open-source, cross-platform JavaScript runtime environment that lets
        developers create servers, web apps, command line tools and scripts.
        It is used to execute javascript unit tests through jest package.

Python
        A Python interpreter is required, version 3.13.x

Cython
        Cython is a programming language inspired from Python, allowing to
        write Python code with C/C++ declarations. The cython code is then
        translated into C/C++ and compiled as a Python library. It is used in
        Code_TYMPAN to interface Python code with TYMPAN C++ libraries. A version
        3.0 or above is required.

Shapely
        Shapely is a Python package for manipulation and analysis of planar
        geometric objects. Version 2.1.2 is currently used.

SWIG
        Swig is a software development tool that connects programs written in
        C and C++ with a variety of high-level programming languages.
        It is needed by the 3rd party dependency *CGAL-bingings*, to connect
        C++ with Python code.


Documentation dependencies
--------------------------

The C++ source code is annotated and API documentation can be
extracted with Doxygen_ and Graphviz_ whereas this documentation is generated from
reStructuredText_ sources thanks to Sphinx_. 
MathJax_ is used to nicely display equations.

| Graphviz_ and Sphinx_ are installed during the Code_TYMPAN installation procedure (please refer to Building Code_TYMPAN part).
| Doxygen_ need to be installed either from your package manager or from the official Windows installer. As there are no specific difficulties, please refer to their official documentations.

.. _reStructuredText: https://docutils.sourceforge.io/rst.html
.. _Sphinx: https://www.sphinx-doc.org/en/master/
.. _Doxygen: https://www.doxygen.nl/index.html
.. _Graphviz: https://www.graphviz.org
.. _MathJax: https://www.mathjax.org/


Windows 11
==========

| This section deals with the installation on Windows 11 using the native Microsoft Visual Studio development
| environment, which is assumed to be already installed.
| At this time, the community edition can be used for free as Code_TYMPAN is an open-source project.
| It can be downloaded at https://visualstudio.microsoft.com/fr/thank-you-downloading-visual-studio/?sku=Community&rel=16
|
| **IMPORTANT** : At this time, Code_TYMPAN is compatible with MSVC 2022, so you must have this toolset installed and selected
| on your Visual Studio environment.
| This corresponds with CMake variable -DCMAKE_GENERATOR_TOOLSET=v143
|
| Installation using MinGW / MSYS has not been attempted and is not supported for now.
| (note that this toolchain is supported by CMake but that Boost is **officially unsupported** on MinGW / MSYS) !

*NB* : Since 4.2, Code_TYMPAN targets 64 bit CPU architecture.

To install the development environment for Windows, here are the various installers
to download and execute.

Boost

        Binary versions of Boost are available on SourceForge.
        For instance, you can download the 64 bits 1.86.0 Boost version for Visual Studio 2022 toolchain at
        https://sourceforge.net/projects/boost/files/boost-binaries/1.86.0/boost_1_86_0-msvc-14.3-64.exe/download
        Adapt version and toolchain to your needs.
        See `Installing Boost`_ for detailed instructions.

Qt 6

        Download the installer for the Qt at https://www.qt.io/download-thank-you?hsLang=en
        Currently Qt 6.8.3 is used with msvc2022_64 toolchain.
        In the Qt installer, select the Qt WebEngine extension.
        Also in the Qt installer, the following additional libraries must be installed : Qt positioning, Qt WebChannel, Qt WebSockets, and Qt WebView.
        Finally, it is preferable for developpers to install Qt Creator.

CMake
        https://cmake.org/download/

CGAL
        See the `Installing CGAL`_ (no real binary installer available on Windows) :
        https://github.com/CGAL/cgal/releases/download/v5.6.1/CGAL-5.6.1.zip

Node.js
        Download the installer at https://nodejs.org/en/download/current
        Currently version 22.12.0 is used.

SWIG
        A prebuilt executable of SWIG can be downloaded at
        https://www.swig.org/download.html
        Download and unzip here C:\\dists\\swigwin-4.0.2, for example.
        Add this folder path to the environment variable PATH.
        Add the following environment variables :

        .. code-block:: winbatch

          set SWIG_DIR=C:\dists\swigwin-4.0.2
          set SWIG_EXECUTABLE=C:\dists\swigwin-4.0.2\swig.exe
          set PATH=C:\dists\swigwin-4.0.2;%PATH%


.. _install-python:

Python environment installation
-------------------------------

| `Download <https://www.python.org/downloads/>`_ and install Python 3.13.x for instance to C:\\Python313.
| **Do not add anything to the PATH.**

Launch a command window (note : do not use powershell for this), adapt and execute following commands to create a 3.13.x python virtual environment :

.. code-block:: winbatch

  c:
  cd \
  cd dists/python
  C:\Python313\python.exe -m venv venv313tympan

  # Activate virtual environment
  C:\dists\python\venv313tympan\Scripts\activate.bat


Installing Boost
----------------
Boost is a mandatory dependency of CGAL. Binary versions of Boost are available
on `SourceForge <https://sourceforge.net/projects/boost/files/boost-binaries/>`_.
The Boost installers install both Boost headers and precompiled libraries.

The procedure below applies for 1.86.0 version of Boost and Visual C++ 2022 :

#. Download and run the file boost_1_86_0-msvc-XX.Y-64.exe (where XX.Y = 14.0 for VC 2015, XX.Y = 14.1 for VC 2017, XX.Y = 14.2 for VC 2019, XX.Y = 14.3 for VC 2022).
   Currently, we suggest to download this
   `file <https://sourceforge.net/projects/boost/files/boost-binaries/1.86.0/boost_1_86_0-msvc-14.3-64.exe/download>`_
#. Extract the files to a new directory, e.g. c:\\Boost\\boost_1_86_0.
#. Set the following two environment variables to point respectively to the path of the libraries and the headers

   .. code-block:: winbatch

     BOOST_LIBRARYDIR = C:\Boost\boost_1_86_0\lib64-msvc-14.3
     BOOST_INCLUDEDIR = C:\Boost\boost_1_86_0

This will help CMake to find Boost.


Installing CGAL
---------------

Installing CGAL is mandatory in order to build Code_TYMPAN.

CGAL depends upon some libraries amongst which Boost and Qt or
build tool such as CMake which are supposed to have already been
installed. It also depends on GMP and MPFR which are available here :
https://github.com/CGAL/cgal/releases/download/v5.6.1/CGAL-5.6.1-win64-auxiliary-libraries-gmp-mpfr.zip

The following procedure is adapted from CGAL documentation named
`Installing from the Source Archive <https://doc.cgal.org/latest/Manual/windows.html#title8>`_

#. Download `CGAL-5.6.1.zip <https://github.com/CGAL/cgal/releases/download/v5.6.1/CGAL-5.6.1.zip>`_
   and extract sources to C:\\dists\\CGAL-5.6.1.
#. Install GMP and MPFR libraries: go to CGAL directory (cd
   C:\\dists\\CGAL-5.6.1),  dowload and extract
   `CGAL-5.6.1-win64-auxiliary-libraries-gmp-mpfr.zip
   <https://github.com/CGAL/cgal/releases/download/v5.6.1/CGAL-5.6.1-win64-auxiliary-libraries-gmp-mpfr.zip>`_
#. Add environment variable CGAL_DIR with value C:\\dists\\CGAL-5.6.1

Configuring Node.js
--------------------

When using a proxy, it is necessary to use an internal repository.
This can be done by editing .npmrc present under user folder.  


.. code-block:: winbatch

  registry=https://%internal_repository%/repository/registry.npmjs.org/
  
Another solution is to configure node to pass through proxy.  
This can also be done by editing .npmrc.

.. code-block:: winbatch

  proxy=http://corporate-proxy.my-company.com:port
  https-proxy=http://corporate-proxy.my-company.com:port

If no proxy is being used, no registry or proxy adress need to be specified.

Building Code_TYMPAN
--------------------
Prerequisite : `Major dependencies`_ must be installed.

First clone git repository to obtain the sources.
In a Git Bash windows or a command windows with git accessible on the PATH :

.. code-block:: winbatch

  C:
  cd \
  mkdir projects
  cd projects
  git clone https://gitlab.com/Code_TYMPAN/code_tympan.git


Second, download and extract `Lightweight dependencies`_.

Third, install python libraries

.. code-block:: winbatch

   pip install -U pip
   pip install -U -r requirements-open-source.txt
   pip install -U -r requirements-bundle.txt
   pip install -U -r requirements-dev.txt

Fourth, check and adapt environment variables which should be set as follow :

.. code-block:: winbatch

  BOOST_INCLUDEDIR = C:\Boost\boost_1_86_0
  BOOST_LIBRARYDIR = C:\Boost\boost_1_86_0\lib64-msvc-14.3
  CGAL_DIR = C:\dists\CGAL-5.6.1
  PATH = C:\Qt\6.8.3\msvc2022_64\bin;C:\Qt\6.8.3\msvc2022_64\plugins\platforms;C:\dists\swigwin-4.0.2;C:\Program Files\doxygen\bin;C:\Program Files (x86)\Graphviz2.38\bin;C:\Boost\boost_1_86_0\lib64-msvc-14.3;%PATH%
  SWIG_DIR = C:\dists\swigwin-4.0.2
  SWIG_EXECUTABLE = C:\dists\swigwin-4.0.2\swig.exe

You can then build Code_TYMPAN in Release or Debug mode using CMake.

Release Build
^^^^^^^^^^^^^
Create build and install repositories :

.. code-block:: winbatch

  cd C:\projects
  mkdir code_tympan_build
  mkdir code_tympan_install

Build and install Code_TYMPAN using CMake :

.. code-block:: winbatch

  cd code_tympan_build
  # Activate python virtual environment
  C:\dists\python\venv313tympan\Scripts\activate.bat
  cmake ..\code_tympan -G "Visual Studio 17 2022" -DCMAKE_GENERATOR_TOOLSET=v143 -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=C:\Qt\6.8.3\msvc2022_64\lib\cmake -DCMAKE_INSTALL_PREFIX=..\code_tympan_install -DCGAL_DIR=%CGAL_DIR% -A x64 >> output_file.txt 2>&1
  # System build output written in output.txt file
  cmake --build . --config Release
  cmake --build . --target install --config Release

Debug Build
^^^^^^^^^^^
Debug and release system builds must be separated.
Create build and install repositories :

.. code-block:: winbatch

  cd C:\projects
  mkdir code_tympan_build_d
  mkdir code_tympan_install_d

Build and install Code_TYMPAN using CMake :

.. code-block:: winbatch

  cd code_tympan_build_d
  # Activate python virtual environment
  C:\dists\python\venv313tympan\Scripts\activate.bat
  cmake ..\code_tympan -G "Visual Studio 17 2022" -DCMAKE_GENERATOR_TOOLSET=v143 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=C:\Qt\6.8.3\msvc2022_64\lib\cmake -DCMAKE_INSTALL_PREFIX=..\code_tympan_install_d -DCGAL_DIR=%CGAL_DIR% -A x64 >> output_file.txt 2>&1
  # System build output written in output.txt file
  cmake --build . --config Debug
  cmake --build . --target install --config Debug


.. _devel-running-tests:

Running Tests
^^^^^^^^^^^^^
.. code-block:: winbatch

  cd code_tympan_build
  # Activate python virtual environment
  C:\dists\python\venv313tympan\Scripts\activate.bat
  SetEnvTympanTests.bat
  set TYMPAN_PYTHON_INTERP=C:\dists\python\venv313tympan\Scripts\python.exe
  ctest -C Release --output-on-failure

Running Linters
^^^^^^^^^^^^^^^
.. code-block:: winbatch

  cd code_tympan_build
  # Activate python virtual environment
  C:\dists\python\venv313tympan\Scripts\activate.bat
  SetEnvTympanTests.bat
  # run lint target
  cmake --build . --target lint

Build Documentation
^^^^^^^^^^^^^^^^^^^

.. code-block:: winbatch

  cd code_tympan_build
  # Activate python virtual environment
  C:\dists\python\venv313tympan\Scripts\activate.bat
  SetEnvTympanTests.bat
  # run doc target
  cmake --build . --target doc


Working with Visual Studio
^^^^^^^^^^^^^^^^^^^^^^^^^^
| To work in Release mode, open Release mode Solution at C:\\projects\\code_tympan_build\\Code_TYMPAN.sln,
| To work in Debug mode, open Debug mode Solution at C:\\projects\\code_tympan_build_d\\Code_TYMPAN.sln.

In Visual Studio, please check build mode (Debug / Release) is consistent
with the solution build mode.

.. note:: More about the CMake build system

  If you want to add components or otherwise change the build system
  for Code_TYMPAN you might want to have a look at :ref:`Build System
  Documentation <build-system>` first.


Installing ``cgal-swig-bindings`` from GitLab Package Registry
--------------------------------------------------------------

On Windows, building ``cgal-swig-bindings`` from sources may take a long time
because the Python bindings are generated with SWIG and compiled against CGAL.

For a standard Release build, Code_TYMPAN can instead use a prebuilt
``cgal-swig-bindings`` package stored in the GitLab Package Registry.

This package currently targets:

.. code-block:: text

   Architecture: x86_64
   Platform: Windows
   Compiler: MSVC 2022 / v143
   Build type: Release
   Python: 3.13
   CGAL: 5.6.1
   SWIG: 4.0.2

The Conan package reference is:

.. code-block:: text

   cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable

Prerequisites
^^^^^^^^^^^^^

The following prerequisites must already be available:

.. code-block:: text

   Python 3.13 virtual environment
   Conan 2
   Access to the Code_TYMPAN GitLab Package Registry
   A GitLab deploy token with read_package_registry permission

If Conan is not already installed in the Python virtual environment, install it
with:

.. code-block:: bat

   C:\dists\python\venv313tympan\Scripts\activate.bat
   python -m pip install "conan>=2,<3"
   conan --version

If this is the first Conan use on the machine, initialize the default Conan
profile:

.. code-block:: bat

   conan profile detect

The generated profile must be checked. For the current Windows Release package,
it should be consistent with:

.. code-block:: text

   [settings]
   arch=x86_64
   build_type=Release
   compiler=msvc
   compiler.runtime=dynamic
   compiler.runtime_type=Release
   compiler.version=194
   os=Windows

Configure the GitLab Conan remote
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Add the Code_TYMPAN GitLab Package Registry as a Conan remote:

.. code-block:: bat

   conan remote add gitlab https://gitlab.com/api/v4/projects/18728059/packages/conan --force

If a corporate proxy is required, define the proxy before running following Conan commands:

.. code-block:: bat

   set HTTP_PROXY=http://proxy.my-company.com:port
   set HTTPS_PROXY=http://proxy.my-company.com:port

Authenticate with a GitLab deploy token:

.. code-block:: bat

   conan remote login gitlab "<GITLAB_DEPLOY_TOKEN_USERNAME>" -p "<GITLAB_DEPLOY_TOKEN_PASSWORD>"

For example, the username may look like:

.. code-block:: text

   gitlab+deploy-token-XXXXXXXX

Do not store the token in a committed file.

Download and deploy the prebuilt package
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Choose a local folder where Conan will deploy the package files.

Example:

.. code-block:: bat

   mkdir C:\projects\code_tympan_conan_deps
   cd /d C:\projects\code_tympan_conan_deps

Then run:

.. code-block:: bat

   conan install ^
     --requires=cgal-swig-bindings/x86_64-windows-msvc143-cgal561-py313-swig402@tympan/stable ^
     --remote=gitlab ^
     --build=never ^
     --deployer-package="cgal-swig-bindings/*" ^
     --deployer-folder=C:\projects\code_tympan_conan_deps\deps ^
     --output-folder=C:\projects\code_tympan_conan_deps\conan-output

After the command, the following directories must exist:

.. code-block:: text

   C:\projects\code_tympan_conan_deps\deps\cgal-swig-bindings\CGAL
   C:\projects\code_tympan_conan_deps\deps\cgal-swig-bindings\bin

The ``CGAL`` directory contains the Python modules and compiled Python extension
modules.

The ``bin`` directory contains the runtime DLLs required by the bindings.

Configure Code_TYMPAN to use the prebuilt bindings
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When configuring Code_TYMPAN with CMake, enable the prebuilt bindings option
and provide the deployed package path:

.. code-block:: bat

   set CGAL_SWIG_BINDINGS_DIR=C:\projects\code_tympan_conan_deps\deps\cgal-swig-bindings

The path must point to the directory containing the ``CGAL`` and ``bin``
subdirectories.

A Release configuration command may look like:

.. code-block:: bat

   cd /d C:\projects\code_tympan_build

   C:\dists\python\venv313tympan\Scripts\activate.bat

   cmake ..\code_tympan ^
     -G "Visual Studio 17 2022" ^
     -DCMAKE_GENERATOR_TOOLSET=v143 ^
     -DCMAKE_BUILD_TYPE=Release ^
     -DCMAKE_PREFIX_PATH=C:\Qt\6.8.3\msvc2022_64\lib\cmake ^
     -DCMAKE_INSTALL_PREFIX=..\code_tympan_install ^
     -DCGAL_DIR=%CGAL_DIR% ^
     -DTYMPAN_USE_PREBUILT_CGAL_BINDINGS=ON ^
     -DTYMPAN_PREBUILT_CGAL_BINDINGS_DIR=%CGAL_SWIG_BINDINGS_DIR% ^
     -A x64

Then build and install Code_TYMPAN as usual:

.. code-block:: bat

   cmake --build . --config Release
   cmake --build . --target install --config Release


Troubleshooting
^^^^^^^^^^^^^^^

If CMake reports that the ``CGAL`` directory is missing, check the value of:

.. code-block:: bat

   echo %CGAL_SWIG_BINDINGS_DIR%

It must be:

.. code-block:: text

   C:\projects\code_tympan_conan_deps\deps\cgal-swig-bindings

The expected directory layout is:

.. code-block:: text

   %CGAL_SWIG_BINDINGS_DIR%\CGAL
   %CGAL_SWIG_BINDINGS_DIR%\bin

If Conan fails with a permission error while generating files such as
``conanbuildenv-release-x86_64.bat``, make sure that ``conan install`` is not
executed from a protected directory such as:

.. code-block:: text

   C:\Windows\System32

Run it from a writable directory instead, for example:

.. code-block:: bat

   cd /d C:\projects\code_tympan_conan_deps

or explicitly use:

.. code-block:: bat

   --output-folder=C:\projects\code_tympan_conan_deps\conan-output

If Conan cannot reach GitLab, check whether a corporate proxy must be
configured:

.. code-block:: bat

   set HTTP_PROXY=http://proxy.my-company.com:port
   set HTTPS_PROXY=http://proxy.my-company.com:port

If authentication fails, check that the deploy token has at least:

.. code-block:: text

   read_package_registry

The ``write_package_registry`` permission is only required to publish or update
the Conan package.


Ubuntu / Debian standard install
================================

This section introduces the installation from the package manager of a
Linux based distribution. It assumes a pretty recent version of the
system on the date of writing, so that most packages are available
from standard package repositories in a compatible version. Here we
use *Debian bookworm*.

You will need to install packages first, then the Code_Tympan third party package.
Build, install, running tests and documentation creation are described.

You will need to add sudo command before apt-get cause packages should be installed as administrator:

.. code-block:: sh

    # Install needed packages:
    sudo apt update
    sudo apt-get install -y software-properties-common cmake swig unzip wget libgmp-dev libmpfr-dev
    sudo apt-get install -y libboost-all-dev libgl1-mesa-dev libglu1-mesa-dev libglut-dev
    sudo apt-get install -y libqt6opengl6 libqt6opengl6-dev qt6-declarative-dev qt6-base-dev qt6-base-private-dev
    sudo apt-get install -y qt6-webchannel-dev qt6-webview-dev doxygen graphviz
    sudo apt-get install -y python3 python3-dev python3-pip python3-tk python3-venv
    sudo apt-get install -y libboost-all-dev
    sudo apt-get install -y nodejs npm
    # CGAL
    wget https://github.com/CGAL/cgal/releases/download/v5.6.1/CGAL-5.6.1.zip -O CGAL-5.6.1.zip
    unzip CGAL-5.6.1.zip
    unzip CGAL-5.6.1.zip
    export CGAL_DIR=$PWD/CGAL-5.6.1
    export CMAKE_PREFIX_PATH=$CGAL_DIR
    # clone CodeTympan
    git clone https://gitlab.com/Code_TYMPAN/code_tympan
    # virtual-env
    python3 -m venv Python3
    . Python3/bin/activate
    mkdir code_tympan_install
    cd code_tympan
    pip install -U pip
    pip install -U -r requirements-open-source.txt
    pip install -U -r requirements-dev.txt
    # Third party install
    cd 3rdparty
    wget https://sourceforge.net/projects/code-tympan/files/3rdparty/Code_TYMPAN-4.7.0_third_party.zip/download -O Code_TYMPAN-4.7.0_third-party.zip
    unzip Code_TYMPAN-4.7.0_third-party.zip
    # build Code Tympan
    cd ..
    mkdir build
    cd build
    $CI_PROJECT_DIR/.env/bin/python3 -c "import numpy, sys; print(sys.executable); print(numpy.__version__); print(numpy.get_include()); print(numpy.__file__)"
    export VENV_PY="$CI_PROJECT_DIR/.env/bin/python3"
    export NUMPY_INC="$($VENV_PY -c 'import numpy; print(numpy.get_include())')"
    echo "NUMPY_INC=$NUMPY_INC"
    export CFLAGS="$CFLAGS -I$NUMPY_INC"
    export CXXFLAGS="$CXXFLAGS -I$NUMPY_INC"
    cmake ../ -DCMAKE_INSTALL_PREFIX=../install -DCGAL_DIR=$CGAL_DIR -DCMAKE_BUILD_TYPE:STRING=Release -DPython_EXECUTABLE=$CI_PROJECT_DIR/.env/bin/python3 -DPython_FIND_VIRTUALENV=ONLY
    make -j4
    # tests
    export TYMPAN_INSTALL_PATH=../code_tympan_install/
    export TYMPAN_PYTHON_INTERP=../Python/bin/python
    source SetEnvTympanTests.sh
    ctest -C Release --output-on-failure
    # doc
    make doc
    # install
    make install