Accessing data in mounted Docker Volumes running in the Jenkins Docker container

I am new to Jenkins but have some experience running docker. I have setup the Jenkins Docker container to run as a privileged root user as follows

docker run -u root --privileged -p 8080:8080 -p 50000:50000 --restart=on-failure -v jenkins_home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock jenkins/jenkins:lts-jdk11

Within the container (exec bash) I have added some custom configurations

apt update apt install docker.io git config --global --add safe.directory "*" 

My Jenkinsfile is

pipeline { agent { docker { image "docker" } } environment { CODECOV_TOKEN = "50dd5c2e-4259-4cfa-97a7-b4429e0d179e" CONTAINER_SUFFIX = "${BUILD_NUMBER}" DOCKER_NETWORK = "variantvalidator_docker_network-$CONTAINER_SUFFIX" } stages { stage("Clone Repository and Create Docker Network") { steps { checkout scm sh 'docker network create $DOCKER_NETWORK' } } stage("Create Directories on Host") { steps { sh 'mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data' sh 'mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share' sh 'mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/seqrepo/' sh 'mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/logs' } } stage("Where am I") { steps { sh 'pwd' sh 'ls -l' } } stage("Build and Run VVTA PostgreSQL") { steps { script { def dockerfile = './db_dockerfiles/vvta/Dockerfile' def vvtaContainer = docker.build("postgres-vvta-${CONTAINER_SUFFIX}", "-f ${dockerfile} ./db_dockerfiles/vvta") vvtaContainer.run("-p 5432:5432 -d --name vvta --network $DOCKER_NETWORK") sh 'echo Building and running VVTA PostgreSQL' } } } stage("Build and Run Validator MySQL") { steps { script { def dockerfile = './db_dockerfiles/vdb/Dockerfile' def validatorContainer = docker.build("mysql-validator-${CONTAINER_SUFFIX}", "-f ${dockerfile} ./db_dockerfiles/vdb") validatorContainer.run("-p 3306:3306 -d --name vdb --network $DOCKER_NETWORK") sh 'echo Building and running Validator MySQL' } } } stage("Build and Run SeqRepo") { steps { script { def dockerfile = './db_dockerfiles/vvsr/Dockerfile' def seqRepoContainer = docker.build("sqlite-seqrepo-${CONTAINER_SUFFIX}", "-f ${dockerfile} ./db_dockerfiles/vvsr") seqRepoContainer.run("--network $DOCKER_NETWORK --privileged -v /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share:/usr/local/share:rw -v /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/seqrepo:/usr/local/share/seqrepo:rw -v /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/logs:/usr/local/share/logs:rw -d") sh 'echo Building and running SeqRepo' } } } stage("Find Seqrepo Mount") { steps { sh 'pwd' sh 'ls -l' sh 'ls -l /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/seqrepo/' } } stage("Build and Run VariantValidator") { steps { script { def dockerfile = './Dockerfile' def variantValidatorContainer = docker.build("variantvalidator-${CONTAINER_SUFFIX}", "-f ${dockerfile} .") variantValidatorContainer.run("--privileged -v /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share:/usr/local/share:rw -v /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/logs:/usr/local/share/logs:rw -v /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/seqrepo:/usr/local/share/seqrepo:rw -d --name variantvalidator-${CONTAINER_SUFFIX} --network $DOCKER_NETWORK") sh 'echo Building and running VariantValidator' } } } stage("Run Pytest and Codecov") { steps { sh 'docker ps' sh 'docker exec variantvalidator-${CONTAINER_SUFFIX} pytest --cov-report=term --cov=VariantValidator/' sh 'docker exec variantvalidator-${CONTAINER_SUFFIX} codecov' } } stage("Cleanup Docker") { steps { sh 'docker stop vvta' sh 'docker rm vvta' sh 'docker stop vdb' sh 'docker rm vdb' sh 'docker stop sqlite-seqrepo-${CONTAINER_SUFFIX}' sh 'docker rm sqlite-seqrepo-${CONTAINER_SUFFIX}' sh 'docker stop variantvalidator-${CONTAINER_SUFFIX}' sh 'docker rm variantvalidator-${CONTAINER_SUFFIX}' sh 'docker network rm $DOCKER_NETWORK' } } } } 

Note: I have assed a couple of ls commands to help with debugging. When I run the build and test, everything sets up and installs, but the volume mounting is not functioning as expected. The data build in stage("Build and Run SeqRepo") is not accessible from the main container, so cannot be mounted further along in the workflow.

The log shows (cropped)

+ mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data [Pipeline] sh + mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share [Pipeline] sh + mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/seqrepo/ [Pipeline] sh + mkdir /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/logs [Pipeline] { (Where am I) [Pipeline] sh + pwd /var/jenkins_home/workspace/VariantValidator_ci [Pipeline] sh + ls -l total 116 ....... drwxr-xr-x 3 root root 4096 Sep 11 10:20 variantvalidator_data 

so I'm happy the directories are being created

The stage("Build and Run SeqRepo") runs and creates the data. The container keeps running.

However the ls of the volume mount suggests that the data cannot be accessed

+ ls -l /var/jenkins_home/workspace/VariantValidator_ci/variantvalidator_data/share/seqrepo/ total 0 

I have used "rw" in the mount commands. Can anyone spot what I'm doing wrong?

I would expect to see directories and files when the ls command is run

This issue suggests a solution, but I am not clear what to do with the information. Same for other relates issues. Hope to write a complete solution here