Goals and general setup of the cloud function configuration

Like in a traditional app, it’s very common that you’ll need sensitive configuration data in a GCP cloud function, e.g. a DB username and password. This article shows a proper way of achieving this by leveraging managed cloud services and an additional Node.js package. The goals of this setup are…

To achieve this, we’ll be using two components for the cloud functions configuration setup:

1. The secure-config package to securely store the complete configuration as an encrypted JSON file. Uses strong AES encryption and standard JSON, works with nearly any runtime environment.
2. GCP Secret Manager for secure storage and passing of the secure-config master key to the cloud function by using an environment variable.

If you wonder that using Secret Manager itself without any addiational package may be sufficient, take a look on the further thoughts.

Steps to implement the configuration management in your cloud function

Install secure-config and create the config files

Install the secure-config packge by running:

npm install @tsmx/secure-config --save

Having this, create a conf subfolder in your project with the configuration files. In the tutorial we’ll create two files, one for local testing purposes without any encrpytion and a production version which will be used in GCP with an encrypted secret.

The unencrypted config file will be conf/config.json with the following simple content:

{
  "secret": "secret-config-value"
}

To create the encrypted production version I recommend to use the secure-config-tool. If you don’t want to install this tool, refer to the secure-config documentation on how to generate encrypted entries without it.

For simplicity I assume you have secure-config-tool installed an we will use 00000000000000000000000000000000 (32x 0) as the encryption key. Having this, create the encrypted configuration for production of the cloud function as following…

cd conf/ 
export CONFIG_ENCRYPTION_KEY=00000000000000000000000000000000
secure-config-tool create -nh -p "secret" ./config.json > ./config-production.json

This will create config-production.json in the conf directory with an encrypted secret, like so:

{ 
  "secret": "ENCRYPTED|a2890c023f1eb8c3d66ee816304e4c30|bd8051d2def1721588f469c348ab052269bd1f332809d6e6401abc3c5636299d
}

Note: By default, GCP will set NODE_ENV=production when you run a cloud function. That’s why the secure-config package will look for conf/config-production.json if you don’t specify something else. For all available options of the secure-config package, refer to the documentation.

To prevent unwanted exposure of sensitive data, use a .gcloudignore file in the root folder of your project to only upload the encrypted production configuration to GCP when deploying. The following lines will tell gcloud to ignore all files in the conf/ folder but the config-production.json.

# don't upload non-production configurations
conf/*
!conf/config-production.json

Make sure to also check uploads to any public code repo in the same way using .gitignore or something similar.

Use the configuration in your code

Use the configuration values in your cloud function code. Here as an ES module in the main function ./index.js:

import secureConfig from '@tsmx/secure-config';
const config = secureConfig();

export const helloGCP = (req, res) => {
 res.json({
    info: 'Hello from GCP cloud functions!',
    secret: config.secret
  });
}

Of course this also works with CommonJS using require. From a code perspective that’s all, next step is to set-up GCP for passing the configurations key to the function.

Store the configuration key in Secret Manager

In your GCP console search for “secret manager” and enable the API if not already done

After secret manager is enabled, click on “CREATE SECRET” on the top and create a new secret with name CONFIG_KEY and a secret value of 00000000000000000000000000000000. After creating the secret you can see it in the list and click on it to view the details.

Directly below the friendly name you can find the secret’s reference which in this case is projects/100374066341/secrets/CONFIG_KEY. This reference will be used later to securely pass the secret as an environment variable to the cloud function.

To verify the value of a secret, click on the three dots behind a version and go to “View secret value”:

Last step is to grant the service account used for function execution the Secret Manager Secret Accessor role so that the secret can be accessed. By default, GCP uses the following accounts to execute cloud functions depending on the generation:

• Gen 1: PROJECT_ID@appspot.gserviceaccount.com
• Gen 2: PROJECT_NUMBER-compute@developer.gserviceaccount.com

For more details on the used IAM account refer to the function identity documentation. Depending on the generation you’ll deploy the function, select the appropriate account under IAM in the console and make sure it has the Secret Manager Secret Accessor role. Add it if necessary by clicking “Edit principal” and then “Add another role”.

That’s it for the secret manager part. The master key needed for decryption of the configuration is now securely stored and ready to use.

Deploy and run the cloud function

The cloud function is now ready to deploy. To do so, we use gcloud functions deploy.

gcloud functions deploy secure-config-function \
--gen2 \
--runtime=nodejs18 \
--region=europe-west3 \
--source=. \
--entry-point=helloGCP \
--set-secrets=CONFIG_ENCRYPTION_KEY=projects/100374066341/secrets/CONFIG_KEY:latest \
--trigger-http \
--allow-unauthenticated

The option --set-secrets=CONFIG_ENCRYPTION_KEY=projects/100374066341/secrets/CONFIG_KEY:latest tells GCP to supply the cloud function with an env var named CONFIG_ENCRYPTION_KEY expected by secure-config with a value of the latest version of the secret projects/100374066341/secrets/CONFIG_KEY. Make sure to replace the secret’s reference with your specific value.

For a complete description of the options refer to the documentation of gcloud functions deploy.

On completion, gcloud will tell you the URL of the successfully deployed function.

...
updateTime: '2023-09-01T20:44:08.493437974Z'
url: https://europe-west3-tsmx-gcp.cloudfunctions.net/secure-config-function

Call this URL to verify the function is working.

curl https://europe-west3-tsmx-gcp.cloudfunctions.net/secure-config-function
{"info":"Hello from GCP cloud functions!","secret":"secure-config-value"}

You should also see the function with a green check mark in your GCP console.

Perfect! The cloud function is deployed and works using a secure configuration management.

Example project at GitHub

A complete example project is available on GitHub.

git clone https://github.com/tsmx/secure-config-cloud-function.git

For easy deployment of the function, a deploy script is provided in package.json. Simply invoke this with npm run. Make sure gcloud is configured properly and you are in the right project.

npm run deploy

Further thoughts

If – and only if – your function is using very few simple configuration values, nesting, structuring, using arrays and managing multiple environments in the configuration are not of interest, I would suggest to stick with Secret Manager only and leave out the usage of the secure-config package.

Normally, at least some of that options are of interest in your project and the use of the package absolutely makes sense. For a full view of the features you’ll get out of that package refer to the documentation.

Happy coding

Useful links

For cloud functions or app engine services using Node.js runtimes GCP will provide a default set of environment variables accessible through process.env. In this article we will explore how this env vars look like for different versions of Node.js in these GCP services.

A very simple project for deploying the needed functions and app engine services to discover these env vars in your own GCP account is also provided.

Cloud functions environment variables

See below for the default process.env variables provided by GCP cloud functions with different Node.js runtimes. Click the link to call a provided function and retrieve the most current result.

{
  LANGUAGE: "en_US:en",
  NODE_OPTIONS: "--max-old-space-size=192",
  K_REVISION: "node20-gen2-get-env-00003-sip",
  PWD: "/workspace",
  FUNCTION_SIGNATURE_TYPE: "http",
  PORT: "8080",
  CNB_STACK_ID: "google.gae.22",
  NODE_ENV: "production",
  CNB_GROUP_ID: "33",
  NO_UPDATE_NOTIFIER: "true",
  HOME: "/root",
  LANG: "en_US.UTF-8",
  K_SERVICE: "node20-gen2-get-env",
  GAE_RUNTIME: "nodejs20",
  SHLVL: "0",
  CNB_USER_ID: "33",
  LC_ALL: "en_US.UTF-8",
  PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  FUNCTION_TARGET: "getEnv",
  K_CONFIGURATION: "node20-gen2-get-env",
  _: "/layers/google.nodejs.functions-framework/functions-framework/node_modules/.bin/functions-framework"
}

{
  LANGUAGE: "en_US:en",
  NODE_OPTIONS: "--max-old-space-size=192",
  K_REVISION: "3",
  PWD: "/workspace",
  FUNCTION_SIGNATURE_TYPE: "http",
  PORT: "8080",
  CNB_STACK_ID: "google.gae.22",
  NODE_ENV: "production",
  CNB_GROUP_ID: "33",
  NO_UPDATE_NOTIFIER: "true",
  HOME: "/root",
  LANG: "en_US.UTF-8",
  GCF_BLOCK_RUNTIME_go112: "410",
  K_SERVICE: "node20-get-env",
  GAE_RUNTIME: "nodejs20",
  SHLVL: "0",
  CNB_USER_ID: "33",
  LC_ALL: "en_US.UTF-8",
  PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  GCF_BLOCK_RUNTIME_nodejs6: "410",
  FUNCTION_TARGET: "getEnv",
  _: "/layers/google.nodejs.functions-framework/functions-framework/node_modules/.bin/functions-framework"
}

{
  LANGUAGE: "en_US:en",
  NODE_OPTIONS: "--max-old-space-size=192",
  K_REVISION: "node18-gen2-get-env-00003-gic",
  PWD: "/workspace",
  FUNCTION_SIGNATURE_TYPE: "http",
  PORT: "8080",
  CNB_STACK_ID: "google.gae.22",
  NODE_ENV: "production",
  CNB_GROUP_ID: "33",
  NO_UPDATE_NOTIFIER: "true",
  HOME: "/root",
  LANG: "en_US.UTF-8",
  K_SERVICE: "node18-gen2-get-env",
  GAE_RUNTIME: "nodejs18",
  SHLVL: "0",
  CNB_USER_ID: "33",
  LC_ALL: "en_US.UTF-8",
  PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  FUNCTION_TARGET: "getEnv",
  K_CONFIGURATION: "node18-gen2-get-env",
  _: "/layers/google.nodejs.functions-framework/functions-framework/node_modules/.bin/functions-framework"
}

{
  LANGUAGE: "en_US:en",
  NODE_OPTIONS: "--max-old-space-size=192",
  K_REVISION: "3",
  PWD: "/workspace",
  FUNCTION_SIGNATURE_TYPE: "http",
  PORT: "8080",
  CNB_STACK_ID: "google.gae.22",
  NODE_ENV: "production",
  CNB_GROUP_ID: "33",
  NO_UPDATE_NOTIFIER: "true",
  HOME: "/root",
  LANG: "en_US.UTF-8",
  GCF_BLOCK_RUNTIME_go112: "410",
  K_SERVICE: "node18-get-env",
  GAE_RUNTIME: "nodejs18",
  SHLVL: "0",
  CNB_USER_ID: "33",
  LC_ALL: "en_US.UTF-8",
  PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  GCF_BLOCK_RUNTIME_nodejs6: "410",
  FUNCTION_TARGET: "getEnv",
  _: "/layers/google.nodejs.functions-framework/functions-framework/node_modules/.bin/functions-framework"
}

{
  NO_UPDATE_NOTIFIER: "true",
  FUNCTION_TARGET: "getEnv",
  NODE_OPTIONS: "--max-old-space-size=192",
  NODE_ENV: "production",
  PWD: "/workspace",
  HOME: "/root",
  DEBIAN_FRONTEND: "noninteractive",
  PORT: "8080",
  K_REVISION: "node16-gen2-get-env-00003-rok",
  K_SERVICE: "node16-gen2-get-env",
  SHLVL: "1",
  GAE_RUNTIME: "nodejs16",
  FUNCTION_SIGNATURE_TYPE: "http",
  PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  K_CONFIGURATION: "node16-gen2-get-env",
  _: "/layers/google.nodejs.functions-framework/functions-framework/node_modules/.bin/functions-framework"
}

{
  GCF_BLOCK_RUNTIME_nodejs6: "410",
  NO_UPDATE_NOTIFIER: "true",
  FUNCTION_TARGET: "getEnv",
  GCF_BLOCK_RUNTIME_go112: "410",
  NODE_OPTIONS: "--max-old-space-size=192",
  NODE_ENV: "production",
  PWD: "/workspace",
  HOME: "/root",
  DEBIAN_FRONTEND: "noninteractive",
  PORT: "8080",
  K_REVISION: "3",
  K_SERVICE: "node16-get-env",
  SHLVL: "1",
  GAE_RUNTIME: "nodejs16",
  FUNCTION_SIGNATURE_TYPE: "http",
  PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  _: "/layers/google.nodejs.functions-framework/functions-framework/node_modules/.bin/functions-framework"
}

App engine environment variables

See below for the default process.env variables provided by GCP app engine with different Node.js runtimes. Click the link to call a provided service and retrieve the most current result.

{
  S2A_ACCESS_TOKEN: "xxxx",
  GAE_MEMORY_MB: "384",
  NO_UPDATE_NOTIFIER: "true",
  LANGUAGE: "en_US:en",
  GAE_INSTANCE: "00c61b117c1d6452581b06dcb5f23b1f1bc9a6c6aaebd47203070aa80a23270580b3af586743534bbd4012aa004c1437350e8cda1dc423b4be",
  HOME: "/root",
  PORT: "8081",
  NODE_OPTIONS: "--max-old-space-size=300 ",
  GAE_SERVICE: "node20-get-env",
  PATH: "/srv/node_modules/.bin/:/workspace/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  CNB_GROUP_ID: "33",
  CNB_USER_ID: "33",
  GAE_DEPLOYMENT_ID: "454320553896815573",
  LANG: "en_US.UTF-8",
  GOOGLE_CLOUD_PROJECT: "tsmx-gcp",
  GAE_ENV: "standard",
  PWD: "/workspace",
  GAE_APPLICATION: "h~tsmx-gcp",
  LC_ALL: "en_US.UTF-8",
  GAE_RUNTIME: "nodejs20",
  GAE_VERSION: "20230819t221144",
  NODE_ENV: "production",
  CNB_STACK_ID: "google.gae.22"
}

{
  S2A_ACCESS_TOKEN: "xxxx",
  GAE_MEMORY_MB: "384",
  LANGUAGE: "en_US:en",
  NO_UPDATE_NOTIFIER: "true",
  GAE_INSTANCE: "00c61b117cf7b9059c648a310d860a5f66a16dc1761710ac82df9a38392a9624cbdebd451f3978bb6cfc0640f5680590beee2d4afe3b214a879c",
  HOME: "/root",
  PORT: "8081",
  NODE_OPTIONS: "--max-old-space-size=300 ",
  GAE_SERVICE: "node18-get-env",
  PATH: "/srv/node_modules/.bin/:/workspace/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  CNB_GROUP_ID: "33",
  CNB_USER_ID: "33",
  GAE_DEPLOYMENT_ID: "454320535647197464",
  LANG: "en_US.UTF-8",
  GOOGLE_CLOUD_PROJECT: "tsmx-gcp",
  GAE_ENV: "standard",
  GAE_APPLICATION: "h~tsmx-gcp",
  LC_ALL: "en_US.UTF-8",
  PWD: "/workspace",
  GAE_RUNTIME: "nodejs18",
  GAE_VERSION: "20230819t221044",
  NODE_ENV: "production",
  CNB_STACK_ID: "google.gae.22"
}

{
  S2A_ACCESS_TOKEN: "xxxx",
  NO_UPDATE_NOTIFIER: "true",
  GAE_MEMORY_MB: "384",
  GAE_INSTANCE: "00c61b117c641ca0a31d2baf0347dc63fc3e870aee7b8707eccebd1f31d5b5372af69bd5178f08349fb3f3ee5ac460efeae28ed842e96fe861",
  HOME: "/root",
  PORT: "8081",
  NODE_OPTIONS: "--max-old-space-size=300 ",
  GAE_SERVICE: "node16-get-env",
  PATH: "/srv/node_modules/.bin/:/workspace/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  GAE_DEPLOYMENT_ID: "454320521333239759",
  DEBIAN_FRONTEND: "noninteractive",
  GOOGLE_CLOUD_PROJECT: "tsmx-gcp",
  GAE_ENV: "standard",
  GAE_APPLICATION: "h~tsmx-gcp",
  PWD: "/workspace",
  GAE_RUNTIME: "nodejs16",
  GAE_VERSION: "20230819t220949",
  NODE_ENV: "production"
}

Project for discovering process.env in different services and runtimes

Alongside with this article the gcp-get-env project is provided on GitHub. This simple Node.js solution ships a function and an express application that can be deployed either as a GCP cloud function or an app engine service. Simply use the provided scripts in package.json to deploy as a function or service with different runtimes.

For that you’ll need an active GCP account and a configured and ready-to-go gcloud CLI on your machine. For details on the installation & configuration see here.

The following deployment scripts are provided in package.json. Simply call npm run [scriptname] to deploy.

Please note that the deployed functions and services would be publicly accessable and may cause charges to your GCP account.

Happy coding

Useful links