4

I am trying to setup a codebuild where I have one module that would create the dependencies for other builds. And these other builds will use the artifacts generated by the first build and these build should be ran in parallel. I was looking at documentation of CodeBuild it seems there isn't information there. Sample of my buildspec.yml

version: 0.2 #env: #variables: # key: "value" # key: "value" #parameter-store: # key: "value" # key: "value" #git-credential-helper: yes phases: #install: #If you use the Ubuntu standard image 2.0 or later, you must specify runtime-versions. #If you specify runtime-versions and use an image other than Ubuntu standard image 2.0, the build fails. #runtime-versions: # name: version # name: version #commands: # - command # - command # pre_build: # commands: # - mkdir artifacts # - pwd # - ls build: commands: - cd frontend - npm install - cd .. - cd othermodule - npm install #post_build: #commands: # - cp -a $CODEBUILD_SRC_DIR/frontend/dist. # - command artifacts: files: - package-lock.json - node_modules/* base-directory: frontend #cache: #paths: # - paths
Improve this question

3 Answers 3

Reset to default
7

CodeBuild is used to automate build step, not to automate the whole CICD process. In CodeBuild, you specify buildspec.yml to automate a sequence of steps that need to be performed in that particular build.

If you need to automate sequence of builds then the easiest option that you have is to use CodePipeline where you can create stage for each step in your CICD process. In your case, this would mean that one step (stage) would be CodeBuild action that you have described in your post which would transition into another stage where you can specify another CodeBuild actions and these actions can be specified to take artifacts from the previous step as an input and you can run them in parallel.

So it would look like this

INPUT -> STAGE (perform initial build) -> STAGE (specify multiple build actions in parallel - in console, this can be done by placing them next to each other horizontally instead of vertically)

The other option, without using CodePipeline, would be to use Lambda function with CloudWatch events. CodeBuild publishes event once the build is done. You can subscribe Lambda function to that event and write a code that would execute following builds as needed.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

My problem is my input not supported by AWS since the source code is in Bitbucket. is there anyway to work around this?
There are multiple ways to solve this. You should check this to get better idea - lgallardo.com/2018/09/07/codepipeline-bitbucket
Good explanation! We use it in production but I'd never use that setup again. The modern CI/CD solution like Github CI workflows, Gitlab CI, CircleCI allows you to have a single configuration file for CI and CD. This simplifies a lot. You can parallelize jobs or complete pipelines, share data between them, configure triggers (when a task should start), post deployments hooks, cross-OS builds and much more. In AWS Codebuild you do not even have a git context! You are totally enforced to lock in CodeBuild, CodePipeline, Lambda, and the integration is on a level of “ok, it works”.
I agree. This is the price we pay for using "simple to use" vendor specific solutions. AWS forces their way which, in this particular case, I find quite annoying. I am relatively ok with the lambda solution(s) but visibility into this process (and troubleshooting) is painful (in my opinion).
AWS CodeBuild now supports Bitbucket.
1

From today codebuild support parallel test and splitting of test

https://aws.amazon.com/about-aws/whats-new/2025/01/aws-codebuild-test-splitting-parallelism/

Improve this answer

Comments

0

As @MEMark has pointed out, currently AWS CodePipeline service supports Bitbuket.

One suitable approach to reach Bitbucket from AWS is by providing an Star Connection(Developer Tools > Code Build > Settings > Connections).

Once the connection is created the pipeline can be deployed using for instance following code:

AWSTemplateFormatVersion: "2010-09-09" Description: TBD Parameters: DeployStage: Type: String Description: name of the stage to deploy(dev, test, prod) BranchName: Type: String Description: TBD CodeStarConnectionARN: Type: String Description: ARN of the codestar connection to Bitbucket. Resources: CodeBuildRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: Effect: Allow Principal: Service: codebuild.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/AdministratorAccess PipelineRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Action: 'sts:AssumeRole' Effect: Allow Principal: Service: codepipeline.amazonaws.com Version: '2012-10-17' Policies: - PolicyName: CodePipelineAccess PolicyDocument: Version: '2012-10-17' Statement: - Action: - TBD - ... Effect: Allow Resource: '*' - Effect: Allow Action: - TBD - ... Resource: "*" BuildUi: Type: AWS::CodeBuild::Project Properties: Name: !Join ['', ['build-ui-', !Ref DeployStage]] Description: TBD ServiceRole: !Ref CodeBuildRole Artifacts: Type: CODEPIPELINE Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: aws/codebuild/standard:5.0 EnvironmentVariables: - Name: DEPLOY_STAGE Type: PLAINTEXT Value: !Ref DeployStage - Name: DEPLOY_BRANCH_NAME Type: PLAINTEXT Value: !Ref BranchName Source: Type: CODEPIPELINE Location: https://[email protected]/organization/projectName.git BuildSpec: buildspec.ui.yml SourceVersion: !Ref BranchName Tags: - Key: cost Value: ui_build TimeoutInMinutes: 10 BuildApi: Type: AWS::CodeBuild::Project Properties: Name: !Join ['', ['build-api-', !Ref DeployStage]] Description: TBD ServiceRole: !Ref CodeBuildRole Artifacts: Type: CODEPIPELINE Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_SMALL Image: aws/codebuild/standard:5.0 EnvironmentVariables: - Name: DEPLOY_STAGE Type: PLAINTEXT Value: !Ref DeployStage - Name: DEPLOY_BRANCH_NAME Type: PLAINTEXT Value: !Ref BranchName Source: Type: CODEPIPELINE Location: https://[email protected]/organization/projectName.git BuildSpec:buildspec.api.yml SourceVersion: !Ref BranchName Tags: - Key: cost Value: api_build TimeoutInMinutes: 10 ArtifactStoreBucket: Type: AWS::S3::Bucket Properties: AccessControl: 'BucketOwnerFullControl' VersioningConfiguration: Status: Enabled NamePipeline: Type: AWS::CodePipeline::Pipeline Properties: ArtifactStore: Location: !Ref ArtifactStoreBucket Type: S3 Name: !Join ['', ['pipeline-', !Ref DeployStage]] RoleArn: !GetAtt PipelineRole.Arn Stages: - Name: Source Actions: - Name: Source ActionTypeId: Category: Source Owner: AWS Provider: CodeStarSourceConnection Version: '1' Configuration: ConnectionArn: !Ref CodeStarConnectionARN FullRepositoryId: "organization/projectName" BranchName: !Ref BranchName OutputArtifacts: - Name: project-source RunOrder: '1' - Name: ParallelBuild Actions: - Name: BuildUi ActionTypeId: Category: Build Owner: AWS Version: 1 Provider: CodeBuild OutputArtifacts: - Name: project-build-ui InputArtifacts: - Name: project-source Configuration: ProjectName: !Ref BuildUi RunOrder: 1 - Name: BuildApi ActionTypeId: Category: Build Owner: AWS Version: 1 Provider: CodeBuild OutputArtifacts: - Name: build-api InputArtifacts: - Name: project-source Configuration: ProjectName: !Ref BuildApi RunOrder: 1

Notes:

  • it assumes the repository has this git url: https://[email protected]/organization/projectName.git
  • CodeBuildRole and PipelineRole should be defined properly for the use case
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.