secure-config – easy and secure NodeJS configuration management

by

A convenient npm package to handle multi-environment NodeJS configurations with encrypted secrets and HMAC validation. Works with CommonJS and ESM/ECMAScript.

With secure-config we provide a lean and secure configuration management for NodeJS.

  • Lean: following the KISS principle – minimalistic and focused feature set, easy to use
  • Secure: state-of-the-art encryption and data integrity for sensible configuration data

Just follow some basic guidelines like a common naming convention and you are in the game!

Note: This documentation refers to the latest version of secure-config, for docs on older versions please look here.

Benefits

  • Easy integration with just a few lines of code. Works with CommonJS as well as ESM/ECMAScript.
  • Strong AES-256-CBC encryption. Secured configuration is still a valid, legible JSON.
  • HMAC validation of your configuration to ensure data integrity.
  • Configuration management for different environments (dev, test, prod…)
  • No need to use 3rd party secret stores, no vendor or platform lock-in.
  • Works perfectly on-premise, with Docker, cloud platforms like Google AppEngine etc.

Encryption and HMAC validation of your configurations are totally optional. You can also use secure-config just to manage plain, unencrypted JSON configurations for different environments. Also you can combine different security features for the different environments, e.g. encryption-only for TEST and encryption plus HMAC validation for PROD.

Usage

Suppose you have the following JSON configuration file config.json with secret information about your database connection…

{ 
  "database": {
    "host": "127.0.0.1", 
    "user": "MySecretDbUser",
    "pass": "MySecretDbPass" 
  } 
}

Step 1 – Install secure-config-tool [optional]

[tsmx@localhost ]$ npm i -g @tsmx/secure-config-tool

Note: In the explanation I will use the provided tool secure-config-tool. You can do all that without the tool as it is NodeJS crypto standard, but it’s a lot more easier so…

Step 2 – Get a secret key and export it.

[tsmx@localhost ]$ secure-config-tool genkey
df9ed9002b...
[tsmx@localhost ]$ export CONFIG_ENCRYPTION_KEY=df9ed9002b...

Step 3 – Encrypt your configuration JSON values and generate a new, secure configuration file.

[tsmx@localhost ]$ secure-config-tool create config.json > conf/config.json
[tsmx@localhost ]$ cat conf/config.json
{ 
  "database": {
    "host": "127.0.0.1", 
    "user": "ENCRYPTED|50ceed2f97223100fbdf842ecbd4541f|df9ed9002bfc956eb14b1d2f8d960a11",
    "pass": "ENCRYPTED|8fbf6ded36bcb15bd4734b3dc78f2890|7463b2ea8ed2c8d71272ac2e41761a35" 
  },
  "__hmac": "3023eb8cf76894c0d5c7f893819916d876f98f781f8944b77e87257ef77c1adf"
}

The generated file should be in the conf/ subfolder of your app. For details see naming conventions.

Step 4 – Use your configuration in the code.

// CommonJS
const conf = require('@tsmx/secure-config')();

// ESM
import secureConfig from '@tsmx/secure-config';
const conf = secureConfig();

function MyFunc() {
  let dbHost = conf.database.host; // = '127.0.0.1'
  let dbUser = conf.database.user; // = 'MySecretDbUser'
  let dbPass = conf.database.pass; // = 'MySecretDbPass'
  //...
}

To change the default behaviour or make use of more advanced features like HMAC validation you can pass custom options to the function returned by require/import.

Step 5 – Run your app using the new encrypted configuration.

[tsmx@localhost ]$ export CONFIG_ENCRYPTION_KEY=df9ed9002b...
[tsmx@localhost ]$ node app.js

File name and directory conventions

You can have multiple configuration files for different environments or stages. They are distinguished by the environment variable NODE_ENV. The basic configuration file name is config.json if this environment variable is not present. If it is present, a configuration file with the name config-[NODE_ENV].json is used. An exception will be thrown if no configuration file is found.

To change the default configuration file name or loading multiple configuration files you can pass the prefix option.

All configuration files must be located in a conf/ directory of the current running app, meaning a direct subdirectory of the current working directory (CWD/conf/).

Example project structure

Let’s assume you are working on a typical project where you are developing locally, using automated tests and finally will deploy to a production environment. From a configuration perspective, you’ll have these three stages…

Development stage

  • NODE_ENV: not set
  • Configuration file: conf/config.json

Production stage

  • NODE_ENV: production
  • Configuration file: conf/config-production.json

Test stage, e.g. Jest

  • NODE_ENV: test
  • Configuration file: conf/config-test.json

The final project structure would look like that:

path-to-your-app/
├── conf/
│       ├── config.json
│       ├── config-production.json
│       └── config-test.json
├── app.js
└── package.json

Note: it is a very common practice that NODE_ENV is set to test when running automated tests. Some test libraries like Jest will do that automatically.

Custom options

To retrieve a configuration using all default values and without advanced features, you simply invoke a function after the require/import statement without any argument (empty set of parenthesis after require or simple method call after import).

// CommonJS
const conf = require('@tsmx/secure-config')();

// ESM
import secureConfig from '@tsmx/secure-config';
const conf = secureConfig();

To make use of the more advanced features and customize default values, you can pass an options object to this function call.

const confOptions = {
  keyVariable: 'CUSTOM_CONFIG_KEY',
  hmacValidation: true, 
  hmacProperty: '_signature',
  prefix: 'myconf'
}

// CommonJS
const conf = require('@tsmx/secure-config')(confOptions);

// ESM
import secureConfig from '@tsmx/secure-config';
const conf = secureConfig(confOptions);

The following options are available.

keyVariable

Type: String

Default: CONFIG_ENCRYPTION_KEY

The name of the environment variable containing the key for decrypting configuration values and validating the HMAC. See also options on how to pass the key.

hmacValidation

Type: Boolean 

Default: false

Specifies if the loaded configuration should be validated against a given HMAC. If set to true, secure-config will validate the HMAC of the decrypted configuration content against a given HMAC using the current key. If the validation fails, an exception will be thrown. If it succeeds, the decrypted configuration will be returned.

The given HMAC is retrieved from a configuration file property with the name of hmacProperty, e.g.:

{
  "database": {
    "host": "127.0.0.1",
    "user": "ENCRYPTED|50ceed2f97223100fbdf842ecbd4541f|df9ed9002bfc956eb14b1d2f8d960a11",
    "pass": "ENCRYPTED|8fbf6ded36bcb15bd4734b3dc78f2890|7463b2ea8ed2c8d71272ac2e41761a35"
  },
  "__hmac": "3023eb8cf76894c0d5c7f893819916d876f98f781f8944b77e87257ef77c1adf"
}

Enabling this option is recommended for production environments as it adds more security to your configuration management ensuring the loaded configuration is safe against tampering. Unwanted modifications of any – even unencrypted – entries in your configuration would cause the HMAC validation to fail and throw the error HMAC validation failed.

Please ensure that your stored configuration files have an appropriate HMAC property before enabling this option. Otherwise loading the configuration would always fail. secure-config-tool adds the HMAC by default when creating secured configuration files.

To get more information on how the HMAC creation & validation works internally, please refer to the package object-hmac which is used for that. The HMAC value is created out of the entire configuration data before encryption / after decryption. Also take a look on the short description how secure-config is working under the hood.

hmacProperty

Type: String 

Default: __hmac

The name of the HMAC property in a configuration file to be validated against. Only used when hmacValidation is set tor true.

Example configuration file using a custom HMAC property name:

{
  "database": {
    "host": "127.0.0.1",
    "user": "ENCRYPTED|50ceed2f97223100fbdf842ecbd4541f|df9ed9002bfc956eb14b1d2f8d960a11",
    "pass": "ENCRYPTED|8fbf6ded36bcb15bd4734b3dc78f2890|7463b2ea8ed2c8d71272ac2e41761a35"
  },
  "_signature": "3023eb8cf76894c0d5c7f893819916d876f98f781f8944b77e87257ef77c1adf"
}

Loading the configuration with HMAC validation enabled:

const confOptions = {
    hmacValidation: true, 
    hmacProperty: '_signature'
}

const conf = require('@tsmx/secure-config')(confOptions);

prefix

Type: String 

Default: config

Use this parameter to change the default file name pattern from config-[NODE_ENV].json to [prefix]-[NODE_ENV].json for loading files with deviating names or additional ones. The value of NODE_ENV will be evaluated as described in the naming conventions.

To load multiple configurations, use the following pattern in your code.

const secureConf = require('@tsmx/secure-config');
const config = secureConf();
const myconf = secureConf({ prefix: 'myconf', keyVariable: 'MYCONF_KEY' });

This example will load the default config.json using the the key from environment variable CONFIG_ENCRYPTION_KEY as well as the additional myconf.json using the key from MYCONF_KEY. Note that different configurations should use different encryption keys.

Depending on the value of NODE_ENV the following configuration files will be loaded in this example.

Value of NODE_ENVVariableFilename
not setconfigconf/config.json
myconfconf/myconf.json
productionconfigconf/config-production.json
myconfconf/myconf-production.json
testconfigconf/config-test.json
myconfconf/myconf-test.json

Injecting the decryption key

The key for decrypting the encrypted values is derived from an environment variable named CONFIG_ENCRYPTION_KEY. You can use the keyVariable option to change the name if you need to.

You can set the environment variable whatever way is most suitable. Here are some examples for common use-cases.

Set/export directly in the command line.

export CONFIG_ENCRYPTION_KEY=0123456789qwertzuiopasdfghjkly

Set the key in your launch.json configuration for developing/debugging.

... 
    "env": { "CONFIG_ENCRYPTION_KEY": "0123456789qwertzuiopasdfghjklyxc" },
...

Best and most secure way of passing the decryption key to a cloud function is using GCP Secret Manager. Create a new secret for the key (e.g. CONFIG_KEY) there and pass it as an an environment variable using the --set-secrets option. 

gcloud functions deploy my-cloud-function \
--source=. \
--entry-point=myFunction \
--trigger-http \
--set-secrets=CONFIG_ENCRYPTION_KEY=projects/100374066341/secrets/CONFIG_KEY:latest

Replace the number in the key reference path with your GCP project number. For more detailed instructions take a look at the article about secure configuration management for a cloud function.

Set the key in an environment block in app.yml.

... 
env_variables:
  CONFIG_ENCRYPTION_KEY: "0123456789qwertzuiopasdfghjklyxc"
...

Note: Make sure to not include the productive app.yml deployment descriptor in your code repository to not expose the production key. In general only your deployment managers should have access to this file as it also contains other sensitive and cost-relevant information like instance sizes and scaling options.

Pass the key to the docker run command.

docker run --env CONFIG_ENCRYPTION_KEY=0123456789qwertzuiopasdfghjklyxc MYIMAGE

There’s a complete docker test example available on GitHub.

For testing with Jest I recommend to create a test key and set it globally for all tests in the jest.config.js.

process.env['CONFIG_ENCRYPTION_KEY'] = '0123456789qwertzuiopasdfghjklyxc';
module.exports = { testEnvironment: 'node' };

For more details also have a look at how to configure Jest.

If your NodeJS app using secure-config should run as a systemd service, set the key in the [Service] section of your service file.

...
[Service]

Environment=CONFIG_ENCRYPTION_KEY=0123456789qwertzuiopasdfghjklyxc
WorkingDirectory=/path/to/your/app
...

Note: You should also set the WorkingDirectory in the service configuration so that secure-config can find the configuration files in the conf/ subfolder correctly.

The key length must be 32 bytes! The value set in CONFIG_ENCRYPTION_KEY has to be:

  • a string of 32 characters length, or
  • a hexadecimal value of 64 characters length (= 32 bytes)

Otherwise an error will be thrown.

Examples of valid key strings:

  • 32 byte string: MySecretConfigurationKey-123$%&/
  • 32 byte hex value: 9af7d400be4705147dc724db25bfd2513aa11d6013d7bf7bdb2bfe050593bd0f

Different keys for each configuration environment are strongly recommended.

Working with secure-config-tool

For an easy handling of secure configuration files it is recommended to use the accompanying secure-config-tool. It is an easy command line tool covering all necessary workflows.

To install the tool, simply run the following command with sufficent rights:

[tsmx@localhost ]$ npm i -g @tsmx/secure-config-tool

Please note that using secure-config-tool is not a must. It is absolutely possible to do everything on your own. For details refer to the implementation available in the GitHub repo. All functions used in the tool are either NodeJs standard or open-source.

Obtaining a secret key

First thing you have to do when securing your configuration files is to get a key for encryption and/or HMAC generation. For that, secure-config-tool offers the genkey option.

[tsmx@localhost ]$ secure-config-tool genkey
a0e9a2d6447adf308cabeb64a075f6f3a45041be956c3f7a94de8a28c0df695e

This function uses the NodeJS standard function randomBytes to generate a cryptographically strong 32 bit long hex key.

Keep the generated key secret. Separate keys for different environments are strongly recommended.

For every operation dealing with encryption/decryption or HMAC’s, the key must be present in an environment variable CONFIG_ENCRYPTION_KEY.

[tsmx@localhost ]$ export CONFIG_ENCRYPTION_KEY=a0e9a2d6447adf308cabeb64a075f6f3a45041be956c3f7a94de8a28c0df695e

As a best practice you should save the generated key into a file and export it directly from there.

[tsmx@localhost ]$ secure-config-tool genkey > prod-key.txt
[tsmx@localhost ]$ export CONFIG_ENCRYPTION_KEY=$(cat prod-key.txt)

Initial creation of a secure configuration file

To easily create a secured configuration out of an existing JSON file, secure-config-tool provides the create option. Using the default settings, this will take your file, encrypt the values of properties matching the terms (user, pass, token) and add a HMAC.

[tsmx@localhost ]$ cat config-plain.json
{
  "database": {
    "host": "localhost",
    "username": "secret-db-user",
    "password": "Test123",
    "port": 1521
  },
  "settings": {
    "https": true,
    "attempts": 5
  }
}
[tsmx@localhost ]$ secure-config-tool create config-plain.json > config-production.json
[tsmx@localhost ]$ cat config-production.json
{
  "database": {
    "host": "localhost",
    "username": "ENCRYPTED|143456e2d2e401bde5d63224a88d44bb|312d01644cc65161f77ce2ed47458a60",
    "password": "ENCRYPTED|2bca8b3b444e9061e57f4e288ef8aecd|99453a3e55cf621f492c474c8a1c0c81",
    "port": 1521
  },
  "settings": {
    "https": true,
    "attempts": 5
  },
  "__hmac": "9d68c2082f2e5fc61268a8171872ae1c15bf15b6a6ab0f82e27e40f19cc156bd"
}

Now you’re ready to use the secured configuration file. For more details on how to customize the file generation, please refer to the docs for create.

Adding new encrypted properties

As you work on your project, you will have to add further sensible properties to your configuration file. Let’s assume you want wo add a property schema with a value of secret-db-schema to the database section in the example above.

For that, use secure-config-tool’s encrypt function for generating the encrypted value and put it into the config file.

[tsmx@localhost ]$ secure-config-tool encrypt "secret-db-schema"
ENCRYPTED|56c1cd4685e1405607b28a84fa083881|bf052def52a7544cfe9b50badd9c907c2b7b399fade2d86b3b9715baa79e6efb
[tsmx@localhost ]$ vi config-production.json # copy and paste the new property value
[tsmx@localhost ]$ cat config-production.json
{
  "database": {
    "host": "localhost",
    "username": "ENCRYPTED|143456e2d2e401bde5d63224a88d44bb|312d01644cc65161f77ce2ed47458a60",
    "password": "ENCRYPTED|2bca8b3b444e9061e57f4e288ef8aecd|99453a3e55cf621f492c474c8a1c0c81",
    "schema": "ENCRYPTED|56c1cd4685e1405607b28a84fa083881|bf052def52a7544cfe9b50badd9c907c2b7b399fade2d86b3b9715baa79e6efb",
    "port": 1521
  },
  "settings": {
    "https": true,
    "attempts": 5
  },
  "__hmac": "9d68c2082f2e5fc61268a8171872ae1c15bf15b6a6ab0f82e27e40f19cc156bd"
}

The new encrypted property is now ready to use.

Please note that after any change to a configuration file, an eventually present HMAC property – like in our example above – will be invalid until it is re-calculated. To do so, just read on…

Updating the HMAC

If you make use of the HMAC validation, you must be aware of that any change of the content of a configuration file will change it’s HMAC value and therefore invalidate any existing HMAC property.

For re-calculation of the HMAC to make it valid again, simply use the update-hmac function.

[tsmx@localhost ]$ secure-config-tool update-hmac -o config-production.json

This command will take your configuration file and re-calculate the HMAC property. Any other value, especially encrypted values, will stay untouched.

Please also refer to the update-hmac docs and the description of how the HMAC validation is working.

Test an existing configuration file

With the test function, secure-config-tool offers an easy way to test if a configuration file is valid with regards to:

  • decrypting the content
  • validating the HMAC

To test a file, simply call…

[tsmx@localhost ]$ secure-config-tool test config-production.json
Decryption: PASSED
HMAC:       PASSED

If you are using a wrong key or encrypted entries are malformed, you”ll get an error.

[tsmx@localhost ]$ secure-config-tool test config-production.json
Decryption failed. Please check that the right key is used and the encrypted secret is valid and has the form "ENCRYPTED|IV|DATA"
See the docs under: https://github.com/tsmx/secure-config
Decryption: FAILED
HMAC:       not tested

If you get this error, please check if you are using the right key and if all encrypted entries are valid according to the specification. If necessary, you can re-create them with the create or encrypt function.

For more details, please refer to the test docs.

To check a single encrypted entry, just copy it and use the decrypt function. It will print out the decrypted value.

[tsmx@localhost ]$ secure-config-tool decrypt "ENCRYPTED|143456e2d2e401bde5d63224a88d44bb|312d01644cc65161f77ce2ed47458a60"
secret-db-user

Encrypted configuration entries

Specification

Encrypted configuration entries must always have the form: ENCRYPTED | IV | DATA.

PartDescription
ENCRYPTEDThe prefix ENCRYPTED used to identify configuration values that must be decrypted.
IVThe ciphers initialization vector (IV) that was used for encryption. Hexadecimal value.
DATAThe AES-256-CBC encrypted value. Hexadecimal value.

Example: "ENCRYPTED|aebc07dd97af3f857cb585b4c956661b|ea18ce1feaa5b8cf4ecb471b9b4401da"

Generation

In general, it is recommended to use secure-config-tool to create and maintain your configuration files. Have a look here for some typical workflows.

Anyways, it’s totally fine to use the standard crypto functions from NodeJS. With the following snippet you can create valid encrypted entries for your configuration files:

const crypto = require('crypto');
const algorithm = 'aes-256-cbc';

function encrypt(value) { 
  let iv = crypto.randomBytes(16); 
  let key = Buffer.from('YOUR_KEY_HERE'); 
  let cipher = crypto.createCipheriv(algorithm, key, iv); 
  let encrypted = cipher.update(value); 
  encrypted = Buffer.concat([encrypted, cipher.final()]); 
  return 'ENCRYPTED|' + iv.toString('hex') + '|' + encrypted.toString('hex');
}

Under the hood

This is how secure-config works: when importing and invoking the package via require('@tsmx/secure-config')() it…

  1. Retrieves the decryption key out of the environment variable CONFIG_ENCRYPTION_KEY
  2. Loads the applicable JSON configuration file.
  3. Recursively iterates the loaded JSON and decrypts all encrypted entries that were found.
  4. Optionally, checks the HMAC of the decrypted configuration object.
  5. Returns a simple JSON object with all secrets decrypted.

Important notes / good to know:

  • Decryption is only done in-memory. Decrypted values are never persisted anywhere.
  • The CONFIG_ENCRYPTION_KEY environment variable is always required, even if the config file doesn’t contain encrypted secrets.
  • Recursion depth is not limited, encrypted entries can be used at every object level in the JSON config file.
  • HMAC validation is optional and can be used without any encryption, e.g. if you don’t have sensitive data but want to ensure data integrity anyways.
  • Recursion depth is not limited, encrypted entries can be used at every object level in the JSON config file.
  • At the time of writing only decryption of simple values is supported (no arrays or complete sub-objects. but properties of sub-objects). This covers most of the use-cases as sensitive data like passwords are normally all plain, nonstructured values.

Sample projects

GitHub

To get familiar with the use of secure-config I provided a secure-config-test project on Github.

Docker Hub

For trying out with Docker and Kubernetes a public docker image is also available on Docker-Hub.

Unit tests

The package contains a comprehensive set of unit tests with a very high coverage of the code. To run them, install or clone the package and run the tests via npm:

npm run test

To output the code coverage run:

npm run test-coverage

Also check out the current coverage stats at Coveralls.

Changelog

Version 2.1.0 (13.12.2023)

  • Support for encrypted properties of objects in arrays added, e.g.
    { configArray: [ { key: 'ENCRYPTED|...' }, { key: 'ENCRYPTED|... ' } ] }

Version 2.2.0 (29.03.2024)

  • Support for loading multiple configurations with new option prefix added.

Older versions

This article refers to the current version of secure-config. Documentations for older versions can be found here.

Useful Links