This repository contains docker image to pass HEP-specific data into kubernetes pod. Using init-container kubernetes feature it passes data from HEP-specific backends to another container which contains only computational logic.
+--------------------------------------------------------+ | | | | +-----------+ | +-------------------+ +-------------+ | | | | | | | | | | LHC Grid | | | Data downloader | | Computation | | | Storage | +------------> | (this repo) | +---------> | logic | | | | | | | | | | +-----------+ | +-------------------+ +-------------+ | | | | Init container | | | | | +--------------------------------------------------------+ Kubernetes Pod Consider you want to pass data through job-input directory:
docker run -v `pwd`/bootstrap-example:/bootstrap -v `pwd`/job-input:/input -ti scr4t/kube-data-downloader /bin/bash -c -l "cd bootstrap; ./download.sh" Kubernetes job description would look like this:
apiVersion: batch/v1 kind: Job metadata: name: test-job spec: template: metadata: name: test-job annotations: pod.beta.kubernetes.io/init-containers: '[ { "name": "download-data", "image": "scr4t/kube-data-downloader:latest", "command": ["/bin/bash", "-l", "-c", "cd /bootstrap; source download.sh"], "imagePullPolicy": "Always", "restartPolicy": "Never", "volumeMounts": [ { "name": "bootstrap-volume", "mountPath": "/bootstrap" }, { "name": "input-volume", "mountPath": "/input" } ] } ]' spec: restartPolicy: Never containers: - image: scr4t/ship:01.03.2016 name: test-job imagePullPolicy: Always command: ["/bin/bash", "-l", "-c", "cp /input/56211dbf9e74775867df168f.stderr /output/56211dbf9e74775867df168f.stderr; sleep 1"] # this file is used in bootstrap-example volumeMounts: - mountPath: /output name: output-volume - mountPath: /input name: input-volume volumes: - name: bootstrap-volume hostPath: path: /Users/user/path/to/bootstrap-example - name: input-volume emptyDir: {} - name: output-volume hostPath: path: /path/to/output/storage