Google Secret Manager native implementation in Cloud Functions

Should you use the brand new implementation? 3 methods compared : Usability, security, performances…

Sorry for the poor illustration, my graphic designer is doing a round the world trip

In August 2021, Google Cloud Functions released a native implementation of Google Secret Manager via environment variables or mounted as a volume.

This article is a comparison of the three best ways to manage secrets in Google Cloud Functions.

If you are new to Google Cloud Function, maybe start from the very beginning with this article giving an overview :

Create and deploy a new React app using Firebase

What’s here?

Introduction

Usability (natives vs library)

Performance (natives vs library)

Security (natives vs library)

Implementation

Introduction

Google Cloud Functions are independent functions using independent code in an independent environment.

Great.

What if we have variables shared by more than one function? And what if these variables are kind of sensitive like credentials?

Google Secret Manager is a fully-managed, central, secure, and convenient storage system for secrets and any kind of keys.

Changing a secret in Secret Manager will change it everywhere. More important than that, access can be controlled and access rights can be managed independently.

IF YOU WERE USING ENVIRONMENT VARIABLES TO STORE SECRETS : That’s a very bad practice, KEEP READING

Usability

Native implementation should naturally be preferred.

A package can have version issues or can be updated, compromising your implementation, like this update of @google-cloud/pubsub that cost a lot of hours to me and fellow developers.

With native implementation, versions and availability tests are made by GCP teams.

Not mentioning a package should be downloaded and loaded, making function heavier and slower (see next section Performance).

Additionally, in terms of code readability, using Google Secret Manager library makes your code longer and harder to read, see by yourself :

from google.cloud import secretmanager
client = secretmanager.SecretManagerServiceClient()
NAME_SECRET = "projects/xxx/secrets/medium_secret/versions/latest"
response = client.access_secret_version(request={"name": NAME_SECRET})
MEDIUM_SECRET = response.payload.data.decode("UTF-8")
print("Secret : {}".format(MEDIUM_SECRET))

versus :

import os 
MEDIUM_SECRET = os.getenv('MEDIUM_SECRET')
print("Secret : {}".format(MEDIUM_SECRET))

Performance

For this article, I compared performances between a google cloud function with secrets mounted as a volume and one using secret manager client library.

I was looking for the impact of a library versus a native implementation on cold start time and execution time.

To compare performances, I created 2 functions (“method”). They were both reading 10 different secrets and printing them. Here are the results :

hot and cold : Function is called every 10 seconds for hot and every 30 minutes for cold

inside and outside : Imports and getting the secrets are made inside or outside the main function. Doing this way, cache between executions is either conserved or deleted.

Secret Manager Client Library versus secret mounted as a volume

Surprisingly, the performance gap between the two methods is not so wide. The cold start time is maybe the most important part. Median time to read 10 secrets via a volume is 400ms faster than via a package. This is due to package loading, increasing cold start.

As expected, no matter the method we use, if the cloud function is warmed up and keep its cache, the execution time will be similar.

Using secret as an environment variable

Too lazy to do it, I assume performance are equal or even faster than mounted as a volume.

Security

What is the most secure between native integration (as mounted volume or environment variable) and Secret Manager client library ?

It’s roughly all the same. In terms of permissions, secrets are stored at the same place with equal access. The choice between the three solutions depends on the trust you have in the libraries you use and your developers.

The least secure solution is : exposing the secret as an environment variable.

Using the term “environment variable” is confusing : Generally, environment variable inside a Google Cloud Function is a bad practice. Variables would be in plain text, accessible by anyone who has access to Google Cloud Functions.

BUT, using environment variable via Secret Manager is 10 times more secure than basic env var. Only specific developers, if any, have access to the secret, as it’s store in Secret Manager, you can grant access to selected users.

So why is it less secure than mounting the secret as a volume ?

Environment variable is more accessible to library or dependency you use. A library author can print or log stack trace, which can compromise your env var. (except if GCF team added a security layer!)

A secret mounted as a volume is theoretically less volatile, therefore less vulnerable than env var. But, in the native implementation made by Google Cloud Functions, it’s all the same from the developer point of view. From a library point of view, peaking into a virtual file is harder than a simple process.env.

The most secured solution is the one using Secret Manager client library.

The difference between env var + volume and SM client library is version control (and package logs but you know that now).

Using any of the three solutions, your secrets stored in Secret Manager will never appear in version control system like Git.

But, what about development and tests?

It’s likely that you will have a file with development password stored in your computer. This file can accidentally be pushed to Git (it happened to everyone), compromising all your passwords.

With Secret Manager client library, this is less likely to happen if you use a service account for development.

Implementation

Mandatory for the three methods ⇒ Creating the secret

Using CLI

You need to have gcloud CLI installed or you can launch Cloud Shell in your GCP Console.

If not already done, we have to enable Google Secret Manager :

gcloud services enable secretmanager.googleapis.com cloudfunctions.googleapis.com

Run this command in a terminal to create the secret :

echo -n "A cray cray secret created from command line" | \\
    gcloud secrets create medium_secret \\
      --data-file=- \\
      --replication-policy automatic

The secret is now created, but Cloud Functions can’t have access to it.

⇒ Grant read access to the secret from the cloud function :

Cloud Functions uses the App Engine default service account to authenticate with Secret Manager. For production use, Google recommends that you configure your function to authenticate using a user-managed service account (see more).

gcloud secrets add-iam-policy-binding medium_secret --role roles/secretmanager.secretAccessor --member <serviceAccount:your-project@appspot.gserviceaccount.com>

Using UI

That part is very well explained by the documentation. (go for CLI!)

Time to start with serious stuff!!

Secret Mounted as volume

Function (python)

The path to access the secret is the following one : /secrets/medium_secret

Deployment

gcloud functions deploy medium_secret_volume \
--region=europe-west2 \
--trigger-http \
--entry-point medium_secret_volume \
--runtime python38 \
--set-secrets '/secrets/medium_secret=projects/xxxxxxxxxx/secrets/medium_secret:latest'

Verify the success in the “Variable” tab of your function :

Secret as Environment variable

Function (python)

Deployment

gcloud functions deploy medium_secret_env \
--region=europe-west2 \
--trigger-http \
--entry-point medium_secret_env \
--runtime python38 \
--set-secrets 'MEDIUM_SECRET_ENV=projects/xxxxxxxxxx/secrets/medium_secret:latest'

Verify the success in the “Variable” tab of your function :

Secret Manager client library

Function (python)

And add this line in requirements.txt :

google-cloud-secret-manager==2.0.0

Deployment

gcloud functions deploy medium_secret_package\
--region=europe-west2 \
--trigger-http \
--entry-point medium_secret_volume \
--runtime python38

All these methods should print the secret.

Easy? Hum… Now it is!

Conclusion

It’s time to make a choice…

If you are interested in securing your Cloud Functions, you should definitely add an API Gateway in front of them :

Secure Google Cloud Functions with API Gateway

You can clap 50 times to reward me for my hours reading documentation.

Adios Hippos!