How to create a basic Cloud Build CI/CD Pipeline

GCloud / CICD / CloudBuild / Serverless

Introduction

Cloud Build is a Serverless CI/CD platform offered by Google Cloud Platform. It can natively integrate with other GCP Products and can easily be integrated with Spinnaker also. We would we deploying application on Cloud Run which is also a serverless offering for container deployment. Some key benefits of Cloud Build are: -

  1. Fully serverless
  2. Flexible - We can use custom build steps and pre-created extensions
  3. Native Docker support
  4. Generous Free Tier - Free 120 build minutes per day
  5. Supports both local as well as building in the cloud

Requirements

  1. A Google Cloud Account associated with a billing account
  2. Google Cloud Shell

NOTE You can run the same through local env, but then you would be required to have appropriate packages and environment variables setup

Getting Started

In this demo we will be building a simple Go App as a container image, pushing the image to Google Container Registry and then deploying it on Cloud Run.

Enabling APIs

  1. Cloud Build
  2. Cloud Run
  3. Google Container Registry
  4. Cloud Source Repo
  5. Cloud Shell (recommended)
  6. Also enable Cloud Run in Cloud Build settings, else builds will fail

NOTE I would be using us-central1 (Iowa) for deploying the app

Getting the application for pipeline

You can use any go application here. I would be using a simple Go web-server using Gin Gonic which prints hello world and a visitor counter. Lets start by cloning the application in Cloud Shell.

# Handling multiple GOPATHs
export GOPATH=$(go env GOPATH)
export GOPATH=${GOPATH##*:}
git clone https://github.com/crazystylus/CloudBuild_demo.git
cp -r CloudBuild_demo/Part-1_Getting-Started/ $GOPATH/src/

Testing the application

For testing the application we need to fetch the dependencies.

pushd $GOPATH/src/Part-1_Getting-Started/
go mod init # Required in go 1.15 and above
go get -d -v ./...
go build -o main
popd
$GOPATH/src/Part-1_Getting-Started/main

We can test by clicking on preview. We can refresh the page to see the counter incrementing.

NOTE Go is pre-installed in Cloud Shell. You may skip this step as we will be creating and testing a container image below

Do we need a Dockerfile ?!?

It depends as we can build app through steps by Cloud Build but the end result would be executable artifacts instead of a container image. We need a container image at the end, so I will be writing a simple dockerfile for it. I am using a simple dockerfile, you may try multi-stage dockerfile also.

FROM golang:1.14
WORKDIR /go/src/app
COPY . .

RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]

We can check the container image by running below and opening preview in Cloud Shell: -

docker build -t hello-cloudbuild:v1 .
docker run -it -p 8080:8080 hello-cloudbuild:v1

Writing Cloud Build yaml configuration

The pipelines in cloud build are written in YAML/JSON files. The name of the cloud build yaml can be specified in the triggers in the UI, otherwise one may use simple cloudbuild.yaml. Here Cloud Build will be creating container image of our app, then pushing the image to container registry. Finally it will deploy it to Cloud Run. We will be using the below pipeline: -

steps:
  # In this step we are building the container image
  - name: "gcr.io/cloud-builders/docker"
    id: Build
    args:
      - "build"
      - "-t"
      - "gcr.io/$PROJECT_ID/hello-cloudbuild:v1-$SHORT_SHA"
      - "."
  # In this step we will be pushing the image to Google Container Registry
  - name: "gcr.io/cloud-builders/docker"
    id: Push
    args:
      - "push"
      - "gcr.io/$PROJECT_ID/hello-cloudbuild:v1-$SHORT_SHA"
  # In this step we will be deploying the image to Cloud Run
  - name: "gcr.io/cloud-builders/gcloud"
    id: Deploy
    args:
      - "run"
      - "deploy"
      - "hello-cloudbuild"
      - "--image"
      - "gcr.io/$PROJECT_ID/hello-cloudbuild:v1-$SHORT_SHA"
      - "--allow-unauthenticated"
      - "--platform"
      - "managed"
      - "--region"
      - "us-central1"
images: ["gcr.io/$PROJECT_ID/hello-cloudbuild:v1-$SHORT_SHA"]

Getting things together

Now as we have the pipeline definition and the dockerfile for building the image, we can start putting the things together.

Create a empty folder, an copy the cloudbuild.yaml, Dockerfile and main.go.

mkdir test-dir
cp cloudbuild.yaml test-dir
cp Dockerfile test-dir
cp main.go test-dir
cd test-dir

Creating Repo and Configuring Cloud Build Trigger

Next we need to create a Cloud Source Repository, then attach the CloudBuild trigger.

# Creating Cloud Source Repo
gcloud source repos create go-hello-cloudbuild

# Creating Cloud Build Trigger
gcloud beta builds triggers create cloud-source-repositories \
--repo=go-hello-cloudbuild \
--tag-pattern='.*' \
--build-config='cloudbuild.yaml'

Next step is initializing the local git repo and adding the remote so that we will be able to push to repo.

# Initializing Git Repo with the code
git init
git add .
git commit -m "Initial Commit"

# Adding Remote to git repo
git config --global credential.'https://source.developers.google.com'.helper gcloud.sh
git remote add origin https://source.developers.google.com/p/${GOOGLE_CLOUD_PROJECT}/r/go-hello-cloudbuild

Triggering the Build

We can trigger the build by creating and pushing a tag to repo.

# Tagging and pushing to git repo
git tag v1
git push -u origin --all
git push -u origin --tags

Next you can visit the Cloud Build service in GCP Console and go to history tab to see the running build. Post build completion, we can see the container image under the Google Container Registry.

Testing the deployed app

Once build is complete, you can either navigate to Cloud Run in Cloud Console or you can run below command to get the endpoint URL for testing the app.

gcloud run services list --platform managed --format json | jq '.[] | select(.metadata.name=="hello-cloudbuild")|.status.url'

Cleaning up the resources

You can run the below script to cleanup all gcp resources.

# Listing triggers
gcloud beta builds triggers list --format json | jq '.[]'

# Getting the trigger id
export TRIGGER_ID=$(gcloud beta builds triggers list --format json | jq -r '.[] | select( .triggerTemplate.repoName=="go-hello-cloudbuild") | .id')
echo $TRIGGER_ID

# Deleting the trigger
gcloud beta builds triggers delete ${TRIGGER_ID} --quiet

# Deleting Source Repository
gcloud source repos delete  go-hello-cloudbuild --quiet

# Deleting Cloud run Service
gcloud run services delete --platform managed --region us-central1 hello-cloudbuild --quiet

# Listing Container Image
gcloud container images list --repository gcr.io/${GOOGLE_CLOUD_PROJECT}

# Deleting Container image
gcloud container images list-tags gcr.io/${GOOGLE_CLOUD_PROJECT}/hello-cloudbuild --format json| jq -r --arg GOOGLE_CLOUD_PROJECT ${GOOGLE_CLOUD_PROJECT} '[.[]| "gcr.io/"+$GOOGLE_CLOUD_PROJECT+"/hello-cloudbuild@"+.digest]| join(" ")' | xargs gcloud container images delete  -q --force-delete-tags

Find Full Code Here https://github.com/crazystylus/CloudBuild_demo/tree/main/Part-1_Getting-Started

 Share!