I want to build a singularity image in GitLab CI. Unfortunately, the official containers fail with:
Running with gitlab-runner 13.5.0 (ece86343) on gitlab-ci d6913e69
Preparing the "docker" executor
Using Docker executor with image quay.io/singularity/singularity:v3.7.0 ...
Pulling docker image quay.io/singularity/singularity:v3.7.0 ...
Using docker image sha256:46d3827bfb2f5088e2960dd7103986adf90f2e5b4cbea9eeb0b0eacfe10e3420 for quay.io/singularity/singularity:v3.7.0 with digest quay.io/singularity/singularity@sha256:def886335e36f47854c121be0ce0c70b2ff06d9381fe8b3d1894fee689615624 ...
Preparing environment
Running on runner-d6913e69-project-2906-concurrent-0 via <gitlab.url>...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in <repo-path>
Checking out 708cc829 as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
Error: unknown command "sh" for "singularity"
immediately at the beginning, when using a job like this:
build-singularity:
image: quay.io/singularity/singularity:v3.7.0
stage: singularity
script:
- build reproduction/pipeline/semrepro-singularity/semrepro-singularity.sif reproduction/pipeline/semrepro-singularity/semrepro-singularity.def
only:
changes:
- reproduction/pipeline/semrepro-singularity/semrepro-singularity.def
- reproduction/pipeline/semrepro-singularity/assets/mirrorlist
- .gitlab/ci/build-semrepo-singularity.yml
artifacts:
paths:
- reproduction/pipeline/semrepro-singularity/semrepro-singularity.sif
expire_in: 1 hour
interruptible: true
For me, it seems like GitLab is trying to use a shell that doesn't exist? How are they supposed to work? In the official example they're using a special version of the docker image called -gitlab, but that unfortunately isn't available anymore. Any ideas? I can't imagine it isn't possible to build singularity containers within CI? Thanks a lot in advance!
EDIT: According to @tsnowlan's answer, overriding the entrypoint fixes the above issue. However, now the build fails with:
singularity build semrepro-singularity.sif semrepro-singularity.def
INFO: Starting build...
INFO: Downloading library image
84.1MiB / 84.1MiB [========================================] 100 % 28.7 MiB/s 0s
ERROR: unpackSIF failed: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: not allowed to create user namespace: exit status 1
FATAL: While performing build: packer failed to pack: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: not allowed to create user namespace: exit status 1
Cleaning up file based variables
ERROR: Job failed: exit code 1
Any ideas?
Thanks a lot! Indeed, this fixes the initial issue that building won't even start! However, it runs into another problem which I experienced before when trying a selfmade docker image. I thought this might be an issue with my custom docker issue, but appareantly it is not. Build fails with
ERROR: unpackSIF failed: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: not allowed to create user namespace. Any ideas?I forgot the sudo. Use that and it should work
Wait no, it should be running as root. That's odd
Aha, yes I think that's it. We run our own gitlab-runners that are run with
--privileged. Though it looks like the shared runners should be too: docs.gitlab.com/13.6/ee/user/gitlab_com/…Ah, if you're running on a managed instance you're gonna be out of luck until they enable privileged mode on the runner. It's a hard require for docker-in-docker or singularity-in-docker