Skip to content
XFELPP

Installation

Packaging Types

Section titled “Packaging Types”

The XFELPP software stack is first and foremost C++, with little, if any, actual Python code (bindings are written in C++). However, we recognize that it is likely that consumers/users of these packages will interact with them mostly through their Python interfaces. As such, all XFELPP software has three principle packaging approaches. Approach 2 is currently used for distributions from this website.

  1. Standalone Wheel: The C++ libraries are bundled into a libs subdirectory of the otherwise standard Python wheel. For example, installing the ncarray package will unpack the wheel into site-packages (at the designated prefix). The C++ libraries will be at site-packages/ncarray/libs. Any header files will be at site-packages/ncarray/include.

  2. Split Wheel: The C++ libraries are built independently from the Python bindings, but are packaged in the associated shared library directory when building the wheel. This procedure is done during the repair stage (e.g. auditwheel on Linux, or delvewheel on Windows). Wheels distributed via the XFELPP simple indices are of this type. For example, the Python package ncarray will be at site-packages/ncarray upon installation. The shared libraries will be at site-packages/ncarray.libs (or similar). Headers will still be under site-packages/ncarray/include.

  3. Dual Packages: The C++ libraries and Python package are built and distributed separately. The libraries will be under $PREFIX/lib, and the Python package in site-packages as normal. This strategy is used for Conda builds, for example. You can refer to the Conda recipe in each software repository for examples.

Installing from the Variant Indices

Section titled “Installing from the Variant Indices”

XFELPP provides its own simple PEP503 compliant package indices. There are two main logistical reasons for distributing this way, as opposed to via PyPI directly:

  1. Size – In particular fat binary GPU builds go beyond the storage quota set by PyPI. Along with reducing the build time, and memory consumption, reducing the size of the libraries is a continual goal overtime. However, it is likely that the builds will never fit within the limits of the PyPI quotas. (Building for multiple CUDA architectures is multiplicative – it’s like packaging multiple packages in one). As such, the alternative indices are provided here. In the future, we may consider requesting a quota increase to list on PyPI as well; however, due to reason 2 below, the self-hosted indices will likely remain regardless.

  2. Variant Management – Each project can currently be built in one of multiple variants or flavors. In particular, you have host (CPU) only builds, along with builds for particular versions of the CUDA toolkit. Hosting all of these variants together in a single index can make depedency management complex for downstream users. (Did you mean to install CUDA 12.8? Or CUDA 13.2? And so on). XFELPP currently provides three indices to separate out the variants explicitly.

Indices

Section titled “Indices”

The following indices are used for wheel distribution. Not all packages are currently available from all indices, though (CUDA builds are active for ncarray for the moment). These will begin to be more widely populated as new releases are generated.

  1. pypi.xfelpp.org/host - CPU Only Builds.

  2. pypi.xfelpp.org/cuda128 - Builds with GPU support built against CUDA 12.8.

  3. pypi.xfelpp.org/cuda132 - Builds with GPU support built against CUDA 13.2.

Installing with pip directly

Section titled “Installing with pip directly”

Installation from the command-line or using pip directly is straightforward. When running pip install simply provide the extra keyword argument --extra-index-url pointing to the index for the variant of interest.

Terminal window
pip install ncarray --extra-index-url https://pypi.xfelpp.org/host

Installing with uv

Section titled “Installing with uv”

In your own project, you can indicate to uv that a dependency should be fetched from a different index. The indices are tried in the order that they are configured. In general, there are two steps when setting up the pyproject.toml file.

  1. Define the index - The custom index can be named. Optionally, the default key can be provided to make it the default index; however, in general this will not be needed.
[[tool.uv.index]]
# Can name the index
name = "XFELPP-Host"
url = "https://pypi.xfelpp.org/host"
  1. Pin the package to the index - once an index is configured, the package can be listed as using it.
[tool.uv.sources]
ncarray = { index = "XFELPP-Host" }


[[tool.uv.index]]
# Can name the index
name = "XFELPP-Host"
url = "https://pypi.xfelpp.org/host"

For more information, refer to the uv documentation.

Configuring Pixi

Section titled “Configuring Pixi”

If you use Pixi for managing your Python packages, pixi can be configured to fetch from additional indices and registries. Pixi is flexible - you can configure this for a specific package in the manifest, as an option in the manifest, or as part of the general pixi configuration.

As an option when listing the dependency

Section titled “As an option when listing the dependency”

When listing the PyPI dependencies in the manifest, an optional index argument can be provided to indicate where that the particular package should be fetched from.

[pypi-dependencies]
ncarray = { version = "*", index = "https://pypi.xfelpp.org/host" }

For more information, refer to the pixi documentation.

pypi-options table in the manifest

Section titled “pypi-options table in the manifest”

To add the index of interest include the following in the pixi manifest pixi.toml:

[pypi-options]
extra-index-urls = ["https://pypi.xfelpp.org/host"]

For more information, refer to the pixi documentation.

Pixi Configuration

Section titled “Pixi Configuration”

You can set this globally using the config.toml file in the respective system configuration folder, XDG compliant folder, the standard OS-dependent config folder (e.g. ~/.config/pixi/config.toml on Linux) or using the folder indicated by the $PIXI_HOME environment variable if it is set.

Alternatively, the configuration can be scoped to a particular project by using the file $PIX_PROJECT/.pixi/config.toml.

To add the index of interest include the following in the config.toml:

[pypi-config]
extra-index-urls = ["https://pypi.xfelpp.org/host"]

For more information, refer to the pixi documentation.