Published on

Getting started with Cloud Deploy and Cloud Run? Here's how, even if Kubernetes isn't your thing

Authors

Google Cloud Run was built as a simpler Container as a Service alternative to Kubernetes. Google Cloud Run enables developers to quickly and easily stand up scalable serverless applications with few drawbacks. Google Cloud Run takes advantage of Skaffold configuration YAML files which are the same config files used by Kubernetes and this enables a smooth transition from Kubernetes to Cloud Run.

Today we're going to cover how to easily integrate Google Cloud Run with Google Cloud Deploy using Skaffold files for non-Kubernetes developers, myself included!

This article assumes you know the basics of Cloud Engineering and Application Development.


Cloud Deploy + Cloud Run = 🔥

  1. Setting up A Cloud Run Service
  2. Using Secrets
  3. Gotcha Skaffold values
  4. Setting up Cloud Build
  5. Cloud Deploy For The Win

Setting up A Cloud Run Service

Unless you're an expert with Skaffold then you probably won't know what properties exist in a Skaffold file or how to customize them. The goal here is to avoid spinning our wheels while attempting to integrate these two amazing services as seasoned application developers. And if you've never used Google Cloud Run then you won't know how to set up a basic service, let's start there.

We're going to start out by utilizing Google Cloud's console to set up a new Cloud Run service and from there we'll let Google Cloud Run do the heavy lifting for us so we end up with a mostly complete Skaffold YAML config file for our service.

  1. Navigate to Google Cloud Run and click on the Create Service button. You'll be presented with a screen that looks like this:
Cloud Run Create Service
  1. Click on the Select button for the Hello World container image. This will take you to the next screen where you'll be able to configure your service. You'll be presented with a screen that looks like this:
Hello Container Image Selection
  1. Modify the authentication setting by selecting Allow unauthenticated invocations so that we can easily try out our service. Scroll down to the bottom of the page and click on the Create button. Your page should look similar to this:
Cloud Run Service Configuration
  1. Once the service is up and running we're presented with the Service Details page. On this page we'll want to click on the YAML tab on the far right:
Service Created
  1. Once on the YAML tab for the service you can simply click the edit button and copy the YAML file to your clipboard. There are a few values that you will need to remove and the gcloud cli command automatically does this for us but if you prefer to use the console then below you'll find the values. Removing all of these lines isn't required but some of them are and the rest will be auto populated during deployment so you don't need to worry about them.

The YAML file will look similar to this:

Google Cloud Console Edit YAML file
hello-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: hello
  namespace: '151830072274'
- selfLink: /apis/serving.knative.dev/v1/namespaces/151830072274/services/hello
- uid: 41da3741-407c-4115-8126-26d00a085bdf
- resourceVersion: AAYHmfItR+A
- generation: 2
- creationTimestamp: '2023-10-12T21:48:06.103866Z'
  labels:
    cloud.googleapis.com/location: us-central1
  annotations:
-   run.googleapis.com/client-name: cloud-console
-   serving.knative.dev/creator: test@goidealsoftware.com
-   serving.knative.dev/lastModifier: test@goidealsoftware.com
-   run.googleapis.com/operation-id: 098192r5-8d76-4d77-b42f-6so97c07d480
    run.googleapis.com/ingress: all
    run.googleapis.com/ingress-status: all
spec:
  template:
    metadata:
      labels:
        client.knative.dev/nonce: 5a07530d-6280-484d-987f-1f5cfd79b696
        run.googleapis.com/startupProbeType: Default
      annotations:
        run.googleapis.com/client-name: cloud-console
        autoscaling.knative.dev/maxScale: '100'
        run.googleapis.com/startup-cpu-boost: 'true'
    spec:
      containerConcurrency: 80
      timeoutSeconds: 300
      serviceAccountName: 151830072274-compute@developer.gserviceaccount.com
      containers:
      - image: us-docker.pkg.dev/cloudrun/container/hello
        ports:
        - name: http1
          containerPort: 8080
        resources:
          limits:
            cpu: 1000m
            memory: 512Mi
        startupProbe:
          timeoutSeconds: 240
          periodSeconds: 240
          failureThreshold: 1
          tcpSocket:
            port: 8080
  traffic:
  - percent: 100
    latestRevision: true

Accessing Secrets

  1. Once the service is created we can now run this gcloud command to get a copy of the Skaffold YAML file that was created for us. If you manually copied and pasted the YAML above then you can skip this step as it is redundant. Doing this will save us a lot of time and effort. The command below will create a file called service.yaml in the current directory. This file will contain all the configuration settings for our service. We'll use this file as our base Skaffold YAML file.
gcloud run services describe hello --format export --region us-central1 --project atribusi > hello-service.yaml

This will create a file called hello-service.yaml in the current directory. This file will contain all the configuration settings for our service. We'll use this file as our base Skaffold YAML file.

hello-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  annotations:
    run.googleapis.com/ingress: all
    run.googleapis.com/ingress-status: all
  labels:
    cloud.googleapis.com/location: us-central1
  name: hello
  namespace: '151830072274'
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: '100'
        run.googleapis.com/client-name: cloud-console
        run.googleapis.com/startup-cpu-boost: 'true'
      labels:
        run.googleapis.com/startupProbeType: Default
    spec:
      containerConcurrency: 80
      containers:
      - image: us-docker.pkg.dev/cloudrun/container/hello
        ports:
        - containerPort: 8080
          name: http1
        resources:
          limits:
            cpu: 1000m
            memory: 512Mi
        startupProbe:
          failureThreshold: 1
          periodSeconds: 240
          tcpSocket:
            port: 8080
          timeoutSeconds: 240
      serviceAccountName: 151837495827-compute@developer.gserviceaccount.com
      timeoutSeconds: 300
  traffic:
  - latestRevision: true
    percent: 100
  1. The file above does not contain secrets or environment variables so lets go ahead and add each of those respectively. The below version contains both an environment variable and a secret. The secret is a reference to a Secret Manager key in Google Cloud and it is exposed as an environment variable. The added code is highlighted in pink.
hello-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  annotations:
    run.googleapis.com/ingress: all
    run.googleapis.com/ingress-status: all
  labels:
    cloud.googleapis.com/location: us-central1
  name: hello
  namespace: '151830072274'
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: '100'
        run.googleapis.com/client-name: cloud-console
        run.googleapis.com/startup-cpu-boost: 'true'
      labels:
        run.googleapis.com/startupProbeType: Default
    spec:
      containerConcurrency: 80
      containers:
      - image: us-docker.pkg.dev/cloudrun/container/hello
        ports:
        - containerPort: 8080
          name: http1
        env:
        - name: ANIMAL_TYPE_BASIC_ENV_VAR
          value: horse
        - name: SUPER_DUPER_SECRET_AS_ENV_VAR
          valueFrom:
            secretKeyRef:
              key: latest
              name: SUPER_DUPER_SECRET_KEY_IN_SECRET_MANAGER
        resources:
          limits:
            cpu: 1000m
            memory: 512Mi
        startupProbe:
          failureThreshold: 1
          periodSeconds: 240
          tcpSocket:
            port: 8080
          timeoutSeconds: 240
      serviceAccountName: 151830072274-compute@developer.gserviceaccount.com
      timeoutSeconds: 300
  traffic:
  - latestRevision: true
    percent: 100

Gotcha Skaffold Values

Because we aren't building our YAML files from scratch Google Cloud Run automatically adds some values to our YAML file that we don't need and will in fact break our deployments. We'll need to remove these values.

Under the spec in the service yaml definition it must have a "name" attribute which operates as the revision name. The value must be unique and it must start with the name of the service followed by any unique characters. Google will automatically generate this value for us and that is the best approach in my opinion. Why deal with the hassle when there is very little upside!

If we don't update the name during deployment then our previous revision will be redeployed and that is obviously not what we want!

I personally ran into this issue a couple of times while building Atribusi and it was a pain to debug. Luckily the logs were helpful but they were located under the Cloud Run service logs not the Cloud Deploy logs so it took a few attempts for me to realize this.

The value is highlighted in pink below.

hello-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  annotations:
    run.googleapis.com/ingress: all
    run.googleapis.com/ingress-status: all
  labels:
    cloud.googleapis.com/location: us-central1
- name: hello
  namespace: '151830072274'
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: '100'
        run.googleapis.com/client-name: cloud-console
        run.googleapis.com/startup-cpu-boost: 'true'
      labels:
        run.googleapis.com/startupProbeType: Default
    spec:
      containerConcurrency: 80
      containers:
      - image: us-docker.pkg.dev/cloudrun/container/hello
        ports:
        - containerPort: 8080
          name: http1
        env:
        - name: ANIMAL_TYPE_BASIC_ENV_VAR
          value: horse
        - name: SUPER_DUPER_SECRET_AS_ENV_VAR
          valueFrom:
            secretKeyRef:
              key: latest
              name: SUPER_DUPER_SECRET_KEY_IN_SECRET_MANAGER
        resources:
          limits:
            cpu: 1000m
            memory: 512Mi
        startupProbe:
          failureThreshold: 1
          periodSeconds: 240
          tcpSocket:
            port: 8080
          timeoutSeconds: 240
      serviceAccountName: 151830072274-compute@developer.gserviceaccount.com
      timeoutSeconds: 300
  traffic:
  - latestRevision: true
    percent: 100

Setting Up Cloud build

What good is a Skaffold YAML file if we can't use it? We'll need to set up a Cloud Build pipeline to deploy our service. We'll need to create a cloudbuild.yaml file in the root of our project.

This file will be used to build our container image and deploy it to Cloud Run. We'll also use this file to set up our Cloud Deploy pipeline. If you need help with the below cloudbuild.yaml file then check out this article.

cloudbuild.yaml
steps:
  - name: 'gcr.io/cloud-builders/git'
    secretEnv: ['SSH_KEY']
    entrypoint: 'bash'
    args:
    - -c
    - |
      echo "$$SSH_KEY" >> /root/.ssh/id_rsa
      chmod 400 /root/.ssh/id_rsa
      cp known_hosts.github /root/.ssh/known_hosts
    volumes:
    - name: 'ssh'
      path: /root/.ssh
  - name: gcr.io/cloud-builders/git
    id: 'fetch-git-repo'
    entrypoint: bash
    args:
    - -c
    - |
      git remote set-url origin 'git@github.com:Go-Ideal-Software-LLC/atribusi'
      git fetch --unshallow
    volumes:
    - name: 'ssh'
      path: /root/.ssh
  - name: node:18
    id: 'npm-install'
    entrypoint: npm
    args: ['install']
  - name: 'gcr.io/cloud-builders/docker'
    id: 'build-hello-docker-image'
    args: ['build', '-t', 'us-central1-docker.pkg.dev/atribusi/atribusi/hello:$BUILD_ID', '-f', 'Dockerfile.hello', '.']
  - name: 'gcr.io/cloud-builders/docker'
    id: 'push-hello-docker-image'
    args: ['push', 'us-central1-docker.pkg.dev/atribusi/atribusi/hello:$BUILD_ID']
    waitFor: ['build-hello-docker-image']
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    id: "deploy-hello-to-dev"
    entrypoint: 'bash'
    args:
    - '-c'
    - |
      gcloud deploy releases create release-$(date '+%Y%m%d-%H%M%S') --skaffold-file skaffold.hello.yaml --delivery-pipeline hello-deployment-pipeline --region us-central1 --images hello_app=us-central1-docker.pkg.dev/atribusi/atribusi/hello:$BUILD_ID
    waitFor: ['push-hello-docker-image']


availableSecrets:
  secretManager:
  - versionName: projects/796283614197/secrets/Github-clone-build-blog/versions/latest
    env: 'SSH_KEY'
options:
  logging: CLOUD_LOGGING_ONLY
  pool:
    name: 'projects/go-ideal-software-websites/locations/us-central1/workerPools/go-ideal-software-websites-pp'

Cloud Deploy For The Win

For Cloud Deploy to work properly we must setup a YAML file for our service and this YAML file will reference the service's YAML that we defined above. This works because we will have different service yaml files for each environment. In our contrived example we only have one environment for our hello app but in an enterprise environment you'd likely have at least two, one for dev and one for prod.

If you notice I highlighted line 9 because that is the actual Google Cloud Run service YAML file that we created above. We'll need to create a new file called skaffold.hello.yaml and add the contents below to it. This file is referenced in the cloudbuild.yaml on line 42. when pushing the latest build to our lowest environment.

skaffold.hello.yaml
apiVersion: skaffold/v4beta6
kind: Config
metadata: 
  name: hello
profiles:
- name: dev
  manifests:
    rawYaml:
    - hello-service.yaml
deploy:
  cloudrun: {}

Now that we have the YAML file setup for our Cloud Run service we will need to manually setup the deployment pipeline and this only ever needs to occur once so it is best to do it manually IMO. Create the below file and run the shell command below the file to create the pipeline.

hello-delivery-pipeline.yaml
apiVersion: deploy.cloud.google.com/v1
kind: DeliveryPipeline
metadata:
  name: hello-deployment-pipeline
description: hello deployment pipeline
serialPipeline:
  stages:
  - targetId: hello-dev
    profiles: [dev]
---

apiVersion: deploy.cloud.google.com/v1
kind: Target
metadata:
  name: hello
description: Hello World
run:
  location: projects/atribusi/locations/us-central1
executionConfigs:
- defaultPool:
    serviceAccount: deployer@atribusi.iam.gserviceaccount.com
  usages:
  - RENDER
  - DEPLOY
create-cloud-deploy-pipeline-hello-service.sh
gcloud deploy apply --file=hello-delivery-pipeline.yaml --region=us-central1 --project=atribusi

After running the above command we can navigate over to Cloud Deploy to see our newly created pipeline. With the pipeline created we can now use our mouse and keyboard to create new deployments to higher environments on the fly!

Cloud Deploy Pipeline Created

The above flow assumes you've set up your IAM policies and service accounts correctly and if you haven't check out reference #2 below.

That is all you need to do to get Cloud Deploy and Cloud Run working together. If you have any questions or comments please feel free to reach out to me on Twitter or LinkedIn.

References:

  1. Github Repo for this example
  2. https://medium.com/google-cloud/deploy-to-cloud-run-from-cloud-deploy-4f83628cf045
  3. https://github.com/cgrotz/blog-examples/blob/main/cloud-deploy-to-cloud-run/skaffold.yaml
Discuss on Twitter