Tim Reynolds

Software & Startups


Securing environment variables in Google Cloud Build

When it came to picking a CI/CD system for building and deploying my first system to Google Cloud Platform (GCP) in the middle of last year (2018), I was happy to see Google Cloud Build (GCB) had just seen a rebirth. Building upon the original version the latest incarnation provides a scalable build service that slots nicely into the security model of GCP and provides an easy initial setup thanks to it's GitHub integration.

Like other GCP services, the documentation provides everything needed to get started. However, one topic that doesn't feel well covered and causes some early confusion for users is how to provide secrets as environment variables, which isn't helped by the differentiation of substitutions vs secrets (encrypted variables).

Substitutions are covered under the configuring builds section of the documentation and allow either default or user-defined values, stored in place text within the cloudbuild.yaml, to be replaced before runtime via token substitution ($, $$, ${}), as outlined in the documentation.

steps:
- name: 'gcr.io/cloud-builders/docker'
  args: ['build',
         '--build-arg',
         'node_version=${_NODE_VERSION_1}',
         '-t',
         'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION}',
         '.']
substitutions:
    _NODE_VERSION: v6.9.1
images: [
    'gcr.io/$PROJECT_ID/build-substitutions-nodejs-${_NODE_VERSION_1}'
]

Secrets (encrypted variables), on the other hand, are hidden away in the documentation under securing builds in a more extensive overview of using encrypted resources.

The high-level approach to encrypted variables is similar to user-defined substitutions in that you'll provide a YAML block which defines custom values but rather than plaintext they're stored as base64-encoded strings, encrypted using GCPs KMS solution. When using an encrypted variable in a build, it must be included in the secretEnv field of the step definition and referenced via $$.

steps:
# Login to provide credentials for the push.
# PASSWORD is decrypted before this step runs.
# Note: You need a shell to resolve environment variables with $$
- name: 'gcr.io/cloud-builders/docker'
  entrypoint: 'bash'
  args: ['-c', 'docker login --username=[MY-USER] --password=$$PASSWORD']
  secretEnv: ['PASSWORD']

secrets:
- kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[KEYRING-NAME]/cryptoKey[KEY-NAME]
  secretEnv:
    PASSWORD: <base64-encoded encrypted Dockerhub password>

Given both user-defined substations and secrets are stored in the cloud build definition file, unlike something like Travis CI, to keep your tokens and other secrets private you'll want to use an encrypted variable while container versions are well suited to custom substations.

If like myself you'd rather see an example over reading about it you can jump straight in with the following steps, however, if you're unfamiliar with Cloud KMS or similar systems I'd take 10 minutes to read the basics.

  1. Create a KeyRing and associated key (CryptoKey)

    gcloud kms keyrings create [KEYRING-NAME] \  --location=global
    
    gcloud kms keys create [KEY-NAME] \
     --location=global \
     --keyring=[KEYRING-NAME] \
     --purpose=encryption
    
  2. Grant Cloud Build service account CryptoKey access

    gcloud kms keys add-iam-policy-binding \
     [KEY-NAME] --location=global --keyring=[KEYRING-NAME] \
     --member=serviceAccount:[SERVICE-ACCOUNT]@cloudbuild.gserviceaccount.com \
     --role=roles/cloudkms.cryptoKeyDecrypter
    

    You'll probably need to poke around in the GCP console to find the correct service account, while you're there it's just as easy to do in the UI.

  3. Encrypt and base64 you're environment variable

    echo -n $MY_SECRET | gcloud kms encrypt \
     --plaintext-file=- \ # - reads from stdin
     --ciphertext-file=- \ # - writes to stdout
     --location=global \
     --keyring=[KEYRING-NAME] \
     --key=[KEY-NAME] | base64
    
  4. Define the value in your cloudbuild.yaml file

    secrets:
    - kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[KEYRING-NAME]/cryptoKey   [KEY-NAME]
      secretEnv:
        MY_SECRET: <base64-encoded encrypted secret>
    

    Replacing the project, keyring and key name as needed

  5. Update the step definition to include reference and decryptions

    - name: 'gcr.io/cloud-builders/docker'
      entrypoint: 'bash'
      args: ['-c', 'docker login --username=[MY-USER] --password=$$PASSWORD']
      secretEnv: ['PASSWORD']
    

For a more in-depth step by step guide check out the latest GCB documentation.