aws codebuild 自定义构建环境和本地构建

参考资料

• Extending AWS CodeBuild with Custom Build Environments

• Docker in custom image sample for CodeBuild

codebuild自定义构建环境

在创建codebuild项目的时候发现

构建环境是 Docker 映像，其中包含构建和测试项目所需的所有内容的完整文件系统

用rust写一个简单的helloworld，添加一段单元测试

#[derive(Debug)]
struct Rectangle {width: u32,height: u32,
}impl Rectangle {fn can_hold(&self, other: &Rectangle) -> bool {self.width > other.width && self.height > other.height}
}pub fn add_tow(a: i32) -> i32 {a + 2
}#[cfg(test)]
mod tests {use super::*;#[test]fn larger_can_hold_smaller() {let larger = Rectangle { width: 8, height: 7 };let smaller = Rectangle { width: 5, height: 1 };assert!(larger.can_hold(&smaller));}use crate::add_tow;#[test]fn if_add_twos() {assert_eq!(4, add_tow(2));}#[test]fn if_add_twos_info() {let result =  add_tow(2);// assert!(5 == result,"failed to add two for 3, get wrong value {}",result);assert_eq!(5,result,"failed to add two for 3, get wrong value {}",result);}
}

创建buildspec.yml配置

version: 0.2phases:install:commands:- echo "install stage"pre_build:commands:- echo "pre_build stage"- cargo versionbuild:commands:- echo "build stage"- cargo buildpost_build:commands:- echo "post_build test stage"- cargo test
artifacts:files:- target/release/rustdemodiscard-paths: yes

创建codebuild项目如下

$ aws codebuild create-project --cli-input-json file://buildfile.json
// buildfile.json
{"name": "test-codebuild-rust","source": {"type": "CODECOMMIT","location": "https://git-codecommit.cn-north-1.amazonaws.com.cn/v1/repos/test-codebuild-rust","gitCloneDepth": 1,"gitSubmodulesConfig": {"fetchSubmodules": false},"insecureSsl": false},"sourceVersion": "refs/heads/master^{8f5b871be5bf92ef70155801b2f35e4af0779632}","artifacts": {"type": "S3","location": "zhaojiew-test","path": "","namespaceType": "NONE","name": "rustdemo","packaging": "ZIP","overrideArtifactName": false,"encryptionDisabled": false},"environment": {"type": "LINUX_CONTAINER","image": "xxxxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/myrust:latest","computeType": "BUILD_GENERAL1_SMALL","environmentVariables": [],"privilegedMode": false,"imagePullCredentialsType": "CODEBUILD"},"serviceRole": "arn:aws-cn:iam::xxxxxxxxx:role/service-role/codebuild-test-codebuild-rust-service-role","timeoutInMinutes": 60,"queuedTimeoutInMinutes": 480,"encryptionKey": "arn:aws-cn:kms:cn-north-1:xxxxxxxxx:alias/aws/s3","logsConfig": {"cloudWatchLogs": {"status": "ENABLED"},"s3Logs": {"status": "DISABLED","encryptionDisabled": false}}
}

开始构建

aws codebuild start-build --project-name test-codebuild-rust

第一次在PROVISIONING阶段报错，由于指定仓库为ecr，需要给codebuild-test-codebuild-rust-service-role权限拉取ecr镜像

BUILD_CONTAINER_UNABLE_TO_PULL_IMAGE: Unable to pull customer's container image. CannotPullContainerError: Error response from daemon: pull access denied for xxxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/myrust, repository does not exist or may require 'docker login': denied: User: CodeBuild

第二次出现POST_BUILD预期错误，即测试失败

COMMAND_EXECUTION_ERROR: Error while executing command: cargo test. Reason: exit status 101

第三次成功build

此外，在选择build角色的时候，可以选择两个不同的角色

这两个角色的区别在于一个是自动创建的servicerole，另一个是该项目的servicerole

测试过程中使用默认servicerole，即使权限足够也会出现以下报错

BUILD_CONTAINER_UNABLE_TO_PULL_IMAGE: Unable to pull customer's container image. CannotPullContainerError: Error response from daemon: pull access denied for 037047667284.dkr.ecr.cn-north-1.amazonaws.com.cn/myrust, repository does not exist or may require 'docker login': denied: User: CodeBuild

codebuild本地构建

https://docs.aws.amazon.com/codebuild/latest/userguide/use-codebuild-agent.html

本地构建意味着虽然codebuild项目仍旧在aws环境中托管，但是实际的构建环境转移到本地

先决条件

• 安装git和docker
• 安装codebuild代理

设置build的环境镜像，可以直接使用托管的ecr或手动按需build，构建环境中支持的命令和环境可以参考dockerbuild内容

aws codebuild doceker image

codebuild托管的构建环境镜像有以下

• standard 4.0
• standard 5.0
• standard 6.0
• amazonlinux2-x86_64-standard:3.0
• amazonlinux2-x86_64-standard:4.0
• amazonlinux2-aarch64-standard:1.0
• amazonlinux2-aarch64-standard:2.0

docker pull public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:3.0

下载codebuild代理image（x86_64 版本），之后使用构建脚本启动，该代理为构建脚本的默认代理，如果没有会自动下载

docker pull public.ecr.aws/codebuild/local-builds:latest

下载构建脚本

构建脚本的具体内容，https://github.com/aws/aws-codebuild-docker-images/blob/master/local_builds/codebuild_build.sh

$ curl -O  https://raw.githubusercontent.com/aws/aws-codebuild-docker-images/master/local_builds/codebuild_build.sh
$ chmod +x codebuild_build.sh
$ ./codebuild_build.sh
The image name flag (-i) must be included for a build to run
The artifact directory (-a) must be included for a build to run
# -i 指定build的image环境
# -a 指定构建的输出目录
# -s 需要构建项目目录，默认为当前目录
# -l 覆盖默认的代理镜像

在本地新建maven项目并创建buildspec.yml

version: 0.2phases:install:runtime-versions:java: corretto8pre_build:commands:- echo Initializing environmentbuild:commands:- echo Build started on `date`- mvn compilepost_build:commands:- echo Build completed on `date`- mvn package
artifacts:files:- appspec.yml- scripts/**/*- target/unicorn-web-project.wardiscard-paths: no

开始构建

./codebuild_build.sh -i public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:3.0 -a artificats

构建输出如下

docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:3.0" -e "ARTIFACTS=/home/ec2-user/efs/test-build-javaweb/artificats" -e "SOURCE=/home/ec2-user/efs/test-build-javaweb" -e "INITIATOR=ec2-user" public.ecr.aws/codebuild/local-builds:latestRemoving network agent-resources_default
Removing volume agent-resources_source_volume
Removing volume agent-resources_user_volume
Creating network "agent-resources_default" with the default driver
Creating volume "agent-resources_source_volume" with local driver
Creating volume "agent-resources_user_volume" with local driver
Pulling agent (amazon/aws-codebuild-local:latest)...
latest: Pulling from amazon/aws-codebuild-local
Pulling build (public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:3.0)...
3.0: Pulling from codebuild/amazonlinux2-x86_64-standard
Creating agent-resources_agent_1 ... done
Creating agent-resources_build_1 ... done
Attaching to agent-resources_agent_1, agent-resources_build_1
agent_1  | [Container] 2023/02/17 06:09:22 Waiting for agent ping
agent_1  | [Container] 2023/02/17 06:09:22 Waiting for DOWNLOAD_SOURCE
agent_1  | [Container] 2023/02/17 06:09:24 Phase is DOWNLOAD_SOURCE
agent_1  | [Container] 2023/02/17 06:09:24 CODEBUILD_SRC_DIR=/codebuild/output/src116956819/src
agent_1  | [Container] 2023/02/17 06:09:24 yamlDoc
agent_1  | [Container] 2023/02/17 06:09:24 YAML location is /codebuild/output/srcDownload/src/buildspec.yml
agent_1  | [Container] 2023/02/17 06:09:24 No commands found for phase name: install
agent_1  | [Container] 2023/02/17 06:09:24 Processing environment variables
agent_1  | [Container] 2023/02/17 06:09:24 Running command echo "Installing corretto(OpenJDK) version 8 ..."