29

I want to define a pipeline to compile, deploy to target and test my project.

This should happen in two distinct ways: an incremental (hopefully fast) build at each commit and a full build scheduled at night.

The following .gitlab-ci.yml has all jobs marked "manual" for testing purposes.

stages: - build - deploy - test variables: BUILD_ARTIFACTS_DIR: "artifacts" build-incremental: timeout: 5h stage: build script: - echo "Building" - ./ci/do-prep - echo "done." artifacts: paths: - $BUILD_ARTIFACTS_DIR/ variables: BUILD_TOP_DIR: "/workspace/builds" tags: - yocto when: manual build-nightly: timeout: 5h stage: build script: - echo "Building" - ./ci/do-prep - echo "done." artifacts: paths: - $BUILD_ARTIFACTS_DIR/ tags: - yocto when: manual deploy: stage: deploy script: - echo "Deploying..." - ./ci/do-deploy - echo "done." tags: - yocto dependencies: - build when: manual test: stage: test script: - echo "Testing..." - ./ci/do-test - echo "done." tags: - yocto dependencies: - deploy when: manual 

This fails with message: deploy job: undefined dependency: build.

How do I explain to GitLab deploy stage needs either build-incremental or build-nightly artifacts?

Later I will have to understand how to trigger build-incremental at commit and build-nightly using a schedule, but that seems to be a different problem.

0

3 Answers 3

7

For your scenario, there are two separate paths through your pipeline depending on the "source": a 'push' or a 'schedule'. You can get the source of the pipeline with the CI_PIPELINE_SOURCE variable. I build these paths separately at first, then combine them:

# First source: push events stages: build deploy test variables: BUILD_ARTIFACTS_DIR: "artifacts" build-incremental: timeout: 5h stage: build script: - echo "Building" - ./ci/do-prep - echo "done." artifacts: paths: - $BUILD_ARTIFACTS_DIR/ variables: BUILD_TOP_DIR: "/workspace/builds" tags: - yocto rules: - if: $CI_PIPELINE_SOURCE == 'push' when: manual - when: never deploy-incremental: stage: deploy script: - echo "Deploying..." - ./ci/do-deploy - echo "done." tags: - yocto needs: ['build-incremental'] rules: - if $CI_PIPELINE_SOURCE == 'push' when: always - when: never test-incremental: stage: test script: - echo "Testing..." - ./ci/do-test - echo "done." tags: - yocto needs: ['deploy-incremental'] rules: - if: $CI_PIPELINE_SOURCE == 'push' when: always - when: never 

In this path, if the source is a push, the build step will run upon manual input, otherwise it will never run. Then, the deploy-incremental step will run automatically (without waiting for other jobs or stages) as long as the source is a push, otherwise it will never run. Finally the test-incremental job will run automatically without waiting for other jobs or stages if it's a push like above.

Now we can build the schedule path:

# Scheduled path: stages: build deploy test variables: BUILD_ARTIFACTS_DIR: "artifacts" build-schedule: timeout: 5h stage: build script: - echo "Building" - ./ci/do-prep - echo "done." artifacts: paths: - $BUILD_ARTIFACTS_DIR/ variables: BUILD_TOP_DIR: "/workspace/builds" tags: - yocto rules: - if: $CI_PIPELINE_SOURCE === 'schedule' when: manual - when: never deploy-schedule: stage: deploy script: - echo "Deploying..." - ./ci/do-deploy - echo "done." tags: - yocto needs: ['build-schedule'] rules: - if $CI_PIPELINE_SOURCE == 'schedule' when: always - when: never test-schedule: stage: test script: - echo "Testing..." - ./ci/do-test - echo "done." tags: - yocto needs: ['deploy-schedule'] rules: - if: $CI_PIPELINE_SOURCE == 'schedule' when: always - when: never 

This works the same way as the push path, but we check to see if the source is schedule.

Now we can combine the two paths:

Combined result: stages: build deploy test variables: BUILD_ARTIFACTS_DIR: "artifacts" build-incremental: timeout: 5h stage: build script: - echo "Building" - ./ci/do-prep - echo "done." artifacts: paths: - $BUILD_ARTIFACTS_DIR/ variables: BUILD_TOP_DIR: "/workspace/builds" tags: - yocto rules: - if: $CI_PIPELINE_SOURCE == 'push' when: manual - when: never build-schedule: timeout: 5h stage: build script: - echo "Building" - ./ci/do-prep - echo "done." artifacts: paths: - $BUILD_ARTIFACTS_DIR/ variables: BUILD_TOP_DIR: "/workspace/builds" tags: - yocto rules: - if: $CI_PIPELINE_SOURCE == 'schedule' when: manual - when: never deploy-incremental: stage: deploy script: - echo "Deploying..." - ./ci/do-deploy - echo "done." tags: - yocto needs: ['build-incremental'] rules: - if $CI_PIPELINE_SOURCE == 'push' when: always - when: never deploy-schedule: stage: deploy script: - echo "Deploying..." - ./ci/do-deploy - echo "done." tags: - yocto needs: ['build-schedule'] rules: - if $CI_PIPELINE_SOURCE == 'schedule' when: always - when: never test-incremental: stage: test script: - echo "Testing..." - ./ci/do-test - echo "done." tags: - yocto needs: ['deploy-incremental'] rules: - if: $CI_PIPELINE_SOURCE == 'push' when: always - when: never test-schedule: stage: test script: - echo "Testing..." - ./ci/do-test - echo "done." tags: - yocto needs: ['deploy-schedule'] rules: - if: $CI_PIPELINE_SOURCE == 'schedule' when: always - when: never 

A pipeline like this is tedious and takes a bit to build, but works great when you have multiple paths/ways to build the project.

Sign up to request clarification or add additional context in comments.

1 Comment

Given how many years have past since this question was asked, is there any updated ways to do this other than repeating every variation of each job?
6

To tell Gitlab that your deploy stage needs certain artifacts from a specific job: Try naming dependencies by job name. In deploy you are defining a dependency with build which is a stage name not the one of the job you want to pick the artifact. Example:

deploy: stage: deploy script: - echo "Deploying..." - ./ci/do-deploy - echo "done." tags: - yocto dependencies: - build-incremental when: manual 

more info and examples here dependencies

1 Comment

This does not answer the main question on how to run a job that depends on either of previous jobs. It just recomends to depend on only one of them.
3

The best way to accomplish this is to use needs more info on needs

Example below:

stages: - 📼 start-main-job - 📎 start-main-job-2 DEV before-main-job: stage: 📼 start-main-job needs: [] when: manual DEV start-main-job: stage: 📼 start-main-job needs: ["DEV before-main-job"] when: on_success DEV before-main-job-2: stage: 📎 start-main-job-2 needs: [] when: manual DEV start-main-job-2: stage: 📎 start-main-job-2 needs: ["DEV before-main-job-2"] when: on_success 

Example pipeline

enter image description here

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.