## page was renamed from Singularity <> == Overview == Singularity or Apptainer allow instantiating an '''alternative operating system environment on our Linux systems'''. In other words: * If you need to develop software in a Scientific Linux 7 environment on your Ubuntu 16.04 desktop and then execute it on a Scientific Linux 6 system, Singularity makes that possible. * If you need to run software from Ubuntu 16.04 (like an up to date PDF viewer) or other software requiring recent versions of some shared libraries (like the Atom editor) on the ancient SL6, Singularity makes that possible. * If you still have software for the discontinued SL5 around and need to run, build, develop or debug it on a supported system, Singularity makes that possible. And all this without significant overhead. '''Sounds like fiction? It's fact. Right now.''' Some caveats do apply though. Credit: Singularity was and is being developed at Berkeley Lab. Please see https://apptainer.org/docs-legacy for upstream information. Apptainer is the new name of Singularity, developed under the umbrella of Linux Foundation. For documentation, see https://apptainer.org/docs . == News == * 2024-05-06 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.3.1|release 1.3.1]] * 2024-04-02 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.3.0|release 1.3.0]] * 2023-12-04 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.2.5|release 1.2.5]] * 2023-11-25 * AlmaLinux 8 image updated to 8.9 * 2023-11-14 * AlmaLinux 9 image updated to 9.3 * 2023-10-20 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.2.4|release 1.2.4]] * 2023-09-19 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.2.3|release 1.2.3]] * 2023-08-14 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.2.2|release 1.2.2]] * 2023-06-23 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.9|release 1.1.9]] * 2023-05-24 * AlmaLinux 8 image updated to 8.8 * 2023-05-11 * AlmaLinux 9 image updated to 9.2 * 2023-05-02 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.8|release 1.1.8]] * 2023-04-13 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.7|release 1.1.7]] * 2023-03-02 * AlmaLinux 9.1 image added * 2023-02-20 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.6|release 1.1.6]] * 2023-01-26 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.5|release 1.1.5]] * 2022-12-22 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.4|release 1.1.4]] * 2022-11-14 * AlmaLinux image updated to 8.7 * 2022-10-27 * Apptainer updated to [[https://github.com/apptainer/apptainer/releases/tag/v1.1.3|release 1.1.3]] * 2022-10-13 * Singularity is replaced by Apptainer [[https://github.com/apptainer/apptainer/releases/tag/v1.1.2|1.1.2]], everywhere except on Ubuntu desktops * 2022-05-12 * AlmaLinux updated to 8.6 * 2022-03-29 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.8.7|release 3.8.7]] * 2022-02-21 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.8.6|release 3.8.6]] * 2021-12-09 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.8.5|release 3.8.5]] * 2021-11-22 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.8.4|release 3.8.4]] * 2021-11-15 * AlmaLinux updated to 8.5 * 2021-10-06 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.8.3|release 3.8.3]] * 2021-09-02 * SL7 and pax images updated to Scientific Linux 7.9 * 2021-07-01 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.8.0|release 3.8.0]] * 2021-06-11 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.7.4|release 3.7.4]] * 2021-06-03 * AlmaLinux updated to 8.4 * 2021-04-27 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.7.3|release 3.7.3]] * 2021-04-12 * AlmaLinux 8.3: initial image of the RHEL8 clone * 2021-03-31 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.7.2|release 3.7.2]] * 2020-12-10 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.7.0|release 3.7.0]] * 2020-12-03 * CentOS7 image updated to 7.9 * 2020-10-26 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.6.4|release 3.6.4]] * 2020-10-05 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.6.3|release 3.6.3]] * 2020-08-11 * Singularity will be updated to [[https://github.com/hpcng/singularity/releases/tag/v3.6.1|release 3.6.1]] * 2020-05-20 * CentOS, SL7 and pax images updated to 7.8 * 2020-03-05 * Singularity will be updated to [[https://github.com/sylabs/singularity/releases/tag/v3.5.3|release 3.5.3]] * 2020-01-29 * Singularity will be updated to [[https://github.com/sylabs/singularity/releases/tag/v3.5.2|release 3.5.2]] * 2019-09-24 * CentOS image updated to 7.7 * 2019-08-30 * SL-based images update to 7.7 * 2019-06-12 * SL-based images update to 7.6 * 2018-12-12 * Singularity will be updated to [[https://github.com/sylabs/singularity/releases/tag/2.6.1|security release 2.6.1]] * 2018-10-29 * SL-based images updated to 6.10 and 7.5 * 2018-09-04 * A new image for CentOS 7.5 was created * 2018-09-03 * Singularity will be updated to release 2.6.0 * 2018-08-01 * A new image pax was created with an environment compatible with the parallel farm * 2018-07-05 * Singularity will be updated to the security release 2.5.2 tonight - no negative impact on users is epected * the SL7 image was updated and no longer provides the "cluster" software/environment * this should enhance compatibility with ordinary WGSs and the (non-pax) batch farm * 2018-06-21 * Gridengine directory /usr/gridengine is bind mounted on SL hosts to provide UGE batch system access * 2018-06-06 * Overlayfs was reenabled * 2018-06-05 * Initial Ubuntu 18.04 image released, needs host newer than SL6 to run * 2018-05-28 * Singularity was updated to release 2.5.1 on all supported systems * 2018-05-01 * Singularity was updated to release 2.5.0 on all supported systems * 2018-04-03 * We had to disable the use of overlayfs for security reasons for the time being. Consequently, bind mounting additional paths will not work unless the mount point exists in the image. This will inhibit CUDA support as described below, at least until we have rolled out images containing /etc/OpenCL/vendors . * 2018-02-21 * Singularity was updated to release 2.4.2 on all supported systems * A set of updated standard images was provided - now in squashfs format which makes them even snappier to start and use == Invocation == There are several ways to instantiate an Apptainer container and run software in it. The two most important ones are: * '''"Execute" an image''' <
> Apptainer images are executable. Executing one of the standard images provided by DV will give you an interactive shell in the container environment: {{{ [wgs03] ~ % lsb_release -d Description: Red Hat Enterprise Linux Server release 6.9 (Santiago) [wgs03] ~ % /project/apptainer/images/U16.img ************************************************************************* * You are now running a shell in a Singularity container providing an * * Ubuntu 16.04 environment. Most things should just work as if you had * * just logged in to such a system, but there are differences. * * * * For more info, please see https:://dvinfo.zeuthen.desy.de/Singularity * ************************************************************************* [wgs03:U16] ~ % lsb_release -d Description: Ubuntu 16.04.2 LTS [wgs03:U16] ~ % }}} Note the modified prompt. It's an attempt to make it more transparent in which environment you're currently working. * '''Run an arbitrary command''' in the context of the image. {{{ [wgs03] ~ % lsb_release -d Description: Red Hat Enterprise Linux Server release 6.9 (Santiago) [wgs03] ~ % apptainer exec /project/apptainer/images/SL5.img lsb_release -d Description: Scientific Linux release 5.11 (Boron) [wgs03] ~ % }}} == Availability == Apptainer containers can be instantiated on any current user accessible Linux system at DESY Zeuthen. In particular, Apptainer is available on all * Linux Desktops * Workgroup Servers * Computes Nodes (Farm & Cluster) The current set of standard images available in `/project/apptainer/images` includes * Scientific Linux 5 (`SL5.sif`) * Scientific Linux 6 (`SL6.sif`) * Scientific Linux 7 (`SL7.sif`) * AlmaLinux 8 (`AL8.sif`) * AlmaLinux 9 (`AL9.sif`) * Ubuntu 16.04 LTS (`U16.sif`) * Ubuntu 18.04 LTS (`U18.sif`) These short image names are symbolic links to the current versions of the standard images for these operating systems. The specific images are being kept around for a few months, but not forever. Note users can copy a current image to any location to make sure this exact version will remain available. Also note '''users can bring their own images''' and thus don't have to rely on the ones provided by DV. Creating those is rather simple and documented upstream. It currently requires root access though, so cannot be done on supported Linux systems in Zeuthen. But have a look at https://singularity-hub.org . Since Release 2.3, it's also possible to import docker images without root privileges. Note that the environment variable `APPTAINER_CACHEDIR` is used to place downloaded image layers, and it defaults to ~/.apptainer. Change the variable to something more sensible to avoid wasting a lot of home space. == Batch system support == In case you want to run your batch jobs directly under the standard images available in `/project/apptainer/images`, it is possible with these lines in the HTCondor submit file {{{ universe=container container_image=/project/apptainer/images/.sif }}} whereas defines the image to be used. == Filesystems shared between Host and Container == Subject to availability on the host: * `$HOME` * `/tmp` * `/afs` * `/lustre` * `/cvmfs` * `/batch` * `/acs`, `/pnfs` (dCache) * `/net` (dCache) * `/nfs` * `/run/user` * `/proc` * `/dev` * `/usr/gridengine` * `$PWD` (the current working directory of the process invoking singularity) Users can specify additional folders on the host to be bind mounted inside the container when the container is started. This shouldn't be required. But if it is, please see `apptainer help run` for info how to do it - and let DV know why you need to do this. == Docker image support == It is recommended to create an image file from docker before running it, for instance: {{{ export APPTAINER_CACHEDIR=/tmp/$(USER)/apptainer-cache apptainer build /tmp/gpt.sif docker://gptdev/notebook apptainer run /tmp/gpt.sif }}} == CUDA support == To run CUDA code in apptainer, the CUDA libraries must be available both inside and outside the container. Apptainer can automatically bind the needed directories into the container like this: {{{ apptainer exec --nv -B /etc/OpenCL/vendors yourprogram }}} == Caveats == * The '''"standard images" are probably not final'''. If you need additional software or configuration, please let us know. * While we made an effort to provide applications running in a Apptainer container with the same set of significant environment variables - and in particular PATH and OS_ARCH should make sense - , '''environment variables inside the container may differ from expectation'''. DV can improve here if required. * While AFS just works, and the credentials (=tokens) inside a container are just the same as outside, the '''AFS sysname is always that of the host'''. There is no way to change this. Which is yet another reason not to rely on AFS sysname magic. To reiterate: using paths like `/afs/zeuthen.desy.de/group/software/@sys/bin` will not work as expected. * '''Automounted filesystems''' on SL6 hosts will '''have to be mounted before the container starts''', thus the mount may have to be triggered outside the container. This affects cvmfs, lustre and d``Cache * Some special applications require compatibility between userland (tied to the container environment) and kernel (tied to the host). In particular: * CUDA * Infini``Band * MPI across hosts, especially over Infini``Band While all of those use cases are objectives of the Apptainer project, actually making them work hasn't been explored yet. In particular, our standard images currently don't have CUDA installed - but see the note on CUDA above. * Setuid/Setgid bits have no effect in the container. We haven't found a case where this causes real problems, but it can be surprising. Affected: * `ping` * `screen` (workaround: use `tmux` instead) * Since the upgrade to Apptainer, the containers run rootless, a Linux kernel mechanism called unprivileged user namespaces is used instead of a setuid root starter executable. This means that files in the container have a different ownership (your user instead of root). If this causes trouble, see the next item: * To work around the limitations of rootless apptainer, an image file can be unpacked before running by using the option `--unsquash`, e.g. for running ancient SL5 firefox: `apptainer exec --unsquash /project/apptainer/images/SL5-management.img firefox` .