How to edit the default container

Which Singularity container a tool will run in is determined by its dependency requirements. If a tool wrapper specifies no package dependencies in its <requirements> block, the tool will run in a default container. The location of the default Singularity container is defined with the singularity_default_container_id setting in the destination_specifications.yaml configuration file. Currently the default container is /srv/galaxy/containers/galaxy-python.sif.

The default container is built by the Ansible playbook¹. The build command can be found in this build.sh script file, and the build is based on the "definition file" galaxy-python.def.

Previous versions of the default container used PIP to install several packages from PYPI and wheels.galaxyproject.org. The packages to include were the ones listed in the requirements.txt file, which was based on the pinned-requirements.txt and conditional-requirements.txt files that Galaxy itself uses.

The current default container does not install many packages but instead sources the virtual environment that Galaxy uses to get direct access to the exact same packages (the "/srv/galaxy" directory containing "/srv/galaxy/venv" is mounted when the container is executed). Python is installed from source inside the container using an approach similar to how Python was installed on the nodes. Perl and CoreUtils are also installed, since these are required by some tools that use this container.

The current definition file for the default Singularity container:

Bootstrap: docker
From: centos:7

# The default container is used by all tools that do not specify any <requirements> in their tool wrappers.
# This usually applies to old and simple tools that do not need any other packages besides those that are used by Galaxy itself.
# Since these tools expect those packages to be available on the server anyway, they have not bothered to specify them as requirements.
# Previous iterations of the default container used PIP to install all the packages that are listed as required dependencies of Galaxy:
#   - https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/dependencies/pinned-requirements.txt
#   - https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/dependencies/conditional-requirements.txt
#
# However, this version of the container does not install anything except for Python 3.8.19 and a few common utilities that are expected on most systems.
# Instead, it activates Galaxy's virtual environment to get access to all these packages. (The venv-directory should be mounted by Singularity).
# The installed Python 3.8.19 is the same as the one that was installed on all the nodes and is referenced by symlinks in the virtual environment.
# Some internal Galaxy tools (like "export history") run code from the Galaxy codebase itself, and for this reason,
# the directory containing Galaxy's codebase (/srv/galaxy/server/lib) is added to the PYTHONPATH on startup.

%post
    # install necessary commands and packages that are required by Galaxy
    yum install -y gcc openssl-devel bzip2-devel libffi-devel zlib-devel xz-devel sqlite-devel ncurses-devel

    # install commands that are needed later in this script
    yum install -y wget make

    # install common utilities that are expected to be available by some tools
    yum install -y coreutils perl

    # update GCC
    yum install -y centos-release-scl
    yum install -y devtoolset-9
    source /opt/rh/devtoolset-9/enable
    echo "Current GCC: $(gcc --version)"

    # download, compile and install Python3.8.19 from source
    cd /usr/src
    wget https://www.python.org/ftp/python/3.8.19/Python-3.8.19.tgz
    tar zxvf Python-3.8.19.tgz
    cd /usr/src/Python-3.8.19
    ./configure --enable-optimizations --enable-loadable-sqlite-extensions
    make
    make install
    rm /usr/src/Python-3.8.19.tgz

%environment
    # activate Galaxy's virtual environment (which should have been mounted) and set PYTHONPATH to include Galaxy's codebase
    source /srv/galaxy/venv/bin/activate
    export PYTHONPATH=/srv/galaxy/server/lib

Not yet: But soon? https://github.com/usegalaxy-no/infrastructure-playbook/issues/63