ERROR: Failed to create user namespace: user namespace disabled - even after disabling setuid in sin-CodePudding

I have to run a singularity.tif file, but it gives me this error:

FATAL [U=1092,P=58809] execStarter() while extracting /net/beegfs/scratch/avanbeelen/beta-version-wgs-pipeline/cnv-tools-scripts/clinSV-0.9/clinsv.sif: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled

After looking into the singularity documentation, this error can be resolved by manually changing the allow setuid = no in the etc/singularity/singularity.conf file. But it still gives me the same error.

If you have already installed Singularity you can disable the setuid flow by setting the option allow setuid = no in etc/singularity/singularity.conf within your installation directory.

Question: Can someone help me to resolve this error such that I can run the singularity.tif?

Important notes:

I am working on HPC cluster where I do not have sudo privileges
I use miniconda to install packages, including singularity
Singularity version = 3.8.6

The code that I run:

    export SINGULARITY_TMPDIR=/net/beegfs/scratch/avanbeelen/singularity-env
    export SINGULARITY_CACHEDIR=/net/beegfs/scratch/avanbeelen/singularity-env
    singularity -d run /net/beegfs/scratch/avanbeelen/beta-version-wgs-pipeline/cnv-tools-scripts/clinSV-0.9/clinsv.sif \
  -i "$input_path/*.bam" \
  -ref $refdata_path \
  -p $output_path

Full error:

DEBUG   [U=1092,P=58809]   persistentPreRun()            Singularity version: 3.8.6
DEBUG   [U=1092,P=58809]   persistentPreRun()            Parsing configuration file /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/etc/singularity/singularity.conf
DEBUG   [U=1092,P=58809]   handleConfDir()               /trinity/home/avanbeelen/.singularity already exists. Not creating.
DEBUG   [U=1092,P=58809]   setValue()                    Updated flag 'tmpdir' value to: /net/beegfs/scratch/avanbeelen/singularity-env
DEBUG   [U=1092,P=58809]   execStarter()                 Saving umask 0002 for propagation into container
VERBOSE [U=1092,P=58809]   execStarter()                 Unprivileged installation: using user namespace
DEBUG   [U=1092,P=58809]   execStarter()                 Checking for encrypted system partition
DEBUG   [U=1092,P=58809]   Init()                        Image format detection
DEBUG   [U=1092,P=58809]   Init()                        Check for sandbox image format
DEBUG   [U=1092,P=58809]   Init()                        sandbox format initializer returned: not a directory image
DEBUG   [U=1092,P=58809]   Init()                        Check for sif image format
DEBUG   [U=1092,P=58809]   Init()                        sif image format detected
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_NODELIST environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CHECKPOINT_IMAGE_DIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_NAME environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding XDG_SESSION_ID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding HOSTNAME environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_TOPOLOGY_ADDR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURMD_NODENAME environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_PRIO_PROCESS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_SRUN_COMM_PORT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding __LMOD_REF_COUNT_MODULEPATH environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding TERM environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SHELL environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_ROOT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding HISTSIZE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_PTY_WIN_ROW environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_QOS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SSH_CLIENT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_TOPOLOGY_ADDR_PATTERN environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding TMPDIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_SHLVL environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_PROMPT_MODIFIER environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_PKG environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CPU_BIND_VERBOSE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_VERSION environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SSH_TTY environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CPU_BIND_LIST environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding USER environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_NNODES environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LS_COLORS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_EXE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEP_NUM_NODES environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SRUN_DEBUG environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOBID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_NTASKS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_LAUNCH_NODE_IPADDR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEP_ID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding _CE_CONDA environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_PREPEND_BLOCK environment variable
VERBOSE [U=1092,P=58809]   SetContainerEnv()             Not forwarding SINGULARITY_TMPDIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_PREFIX_1 environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding _ModuleTable001_ environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding MAIL environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_TASKS_PER_NODE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEP_LAUNCHER_PORT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_WORKING_CLUSTER environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CPUS_PER_TASK environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_ID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_PREFIX environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_SETTARG_CMD environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding PWD environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEPID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_USER environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_SRUN_COMM_HOST environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CPU_BIND_TYPE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LANG environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding MODULEPATH environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_UMASK environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_PTY_WIN_COL environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding _ModuleTable_Sz_ environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_UID environment variable
VERBOSE [U=1092,P=58809]   SetContainerEnv()             Not forwarding SINGULARITY_CACHEDIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_NODEID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_SUBMIT_DIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_CMD environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_NPROCS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_TASK_PID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_DISTRIBUTION environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CPUS_ON_NODE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding _CE_M environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding HISTCONTROL environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_PROCID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_NODELIST environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SHLVL environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_PTY_PORT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_LOCALID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CLUSTER_NAME environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_CPUS_PER_NODE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_GID environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_SUBMIT_HOST environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_GTIDS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding BASH_ENV environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_PARTITION environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_PYTHON_EXE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LOGNAME environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEP_NUM_TASKS environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SSH_CONNECTION environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_ACCOUNT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding MODULESHOME environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_JOB_NUM_NODES environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding CONDA_DEFAULT_ENV environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_SETTARG_FULL_SUPPORT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LESSOPEN environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEP_TASKS_PER_NODE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_FULL_SETTARG_SUPPORT environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_STEP_NODELIST environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding XDG_RUNTIME_DIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding DISPLAY environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_DIR environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding SLURM_CPU_BIND environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding LMOD_COLORIZE environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding BASH_FUNC_module() environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding BASH_FUNC_ml() environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding _ environment variable
DEBUG   [U=1092,P=58809]   SetContainerEnv()             Forwarding USER_PATH environment variable
VERBOSE [U=1092,P=58809]   SetContainerEnv()             Setting HOME=/trinity/home/avanbeelen
VERBOSE [U=1092,P=58809]   SetContainerEnv()             Setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
VERBOSE [U=1092,P=58809]   execStarter()                 User namespace requested, convert image /net/beegfs/scratch/avanbeelen/beta-version-wgs-pipeline/cnv-tools-scripts/clinSV-0.9/clinsv.sif to sandbox
INFO    [U=1092,P=58809]   execStarter()                 Converting SIF file to temporary sandbox...
DEBUG   [U=1092,P=58809]   Init()                        Image format detection
DEBUG   [U=1092,P=58809]   Init()                        Check for sandbox image format
DEBUG   [U=1092,P=58809]   Init()                        sandbox format initializer returned: not a directory image
DEBUG   [U=1092,P=58809]   Init()                        Check for sif image format
DEBUG   [U=1092,P=58809]   Init()                        sif image format detected
DEBUG   [U=1092,P=58809]   extract()                     Excluding /dev directory during root filesystem extraction (non root user)
DEBUG   [U=1092,P=58809]   extract()                     Trying unsquashfs options: [-no-xattrs -r]
DEBUG   [U=1092,P=58809]   unsquashfsSandboxCmd()        Calling wrapped unsquashfs: singularity [-q exec --no-home --no-nv --no-rocm -C --no-init --writable -B /net/beegfs/scratch/avanbeelen/singularity-env/rootfs-2015816184:/image -B /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/unsquashfs:/net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/unsquashfs:ro -B /lib64/libpthread.so.0:/lib64/libpthread.so.0:ro -B /lib64/libm.so.6:/lib64/libm.so.6:ro -B /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/libz.so.1:/net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/libz.so.1:ro -B /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/liblzma.so.5:/net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/liblzma.so.5:ro -B /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/liblzo2.so.2:/net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/liblzo2.so.2:ro -B /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/liblz4.so.1:/net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/liblz4.so.1:ro -B /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/libzstd.so.1:/net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/../lib/libzstd.so.1:ro -B /lib64/libc.so.6:/lib64/libc.so.6:ro -B /lib64/ld-linux-x86-64.so.2:/lib64/ld-linux-x86-64.so.2:ro -B /lib64/librt.so.1:/lib64/librt.so.1:ro /net/beegfs/scratch/avanbeelen/singularity-env/rootfs-2015816184/tmp-rootfs-4287831676 /lib64/ld-linux-x86-64.so.2 /net/beegfs/scratch/avanbeelen/miniconda3/envs/wgs_pipeline/bin/unsquashfs -no-xattrs -r -d /image/root /image/archive-3018451646 ^(.{0}[^d]|.{1}[^e]|.{2}[^v]|.{3}[^\x2f]).*$]
FATAL   [U=1092,P=58809]   execStarter()                 while extracting /net/beegfs/scratch/avanbeelen/beta-version-wgs-pipeline/cnv-tools-scripts/clinSV-0.9/clinsv.sif: root filesystem extraction failed: extract command failed: ERROR  : Failed to create user namespace: user namespace disabled
: exit status 1

CodePudding user response：

Short answer: ask your HPC admins to install singularity.

Singularity relies on the binary being root owned, so an unprivileged conda installation is unlikely to work. In order for the config change you made to have the effect you desire, singularity must be compiled with special flags and the host OS must have the appropriate kernel settings enabled. The details for both are in admin setup documentation.