DocumentationAPI ReferencePlaygroundCommunity

Settings, Environment Variables and Secrets

Core assumptions and management for the settings of Diffgram.

Suggest Edits

Overview

There are multiple ways to load secrets.

During development, if you are using the install script or setting the values in your .env file, that's one of the best ways.

For production, the env should be loaded through a secrets manager, such as built into your CI/CD process. Be sure whatever process you choose enables secret rotation inline with your security posture.

Detail

System settings for Diffgram are loaded from Environment Variables in shared/settings/settings.py.
If you inspect this file you will see that all of the setting are fetched from environment variables.

To adjust these either:

  1. Use a .env file. The Diffgram installer generates this for you. (E.G. for Docker or Kubernetes context)
  2. Use a secret loading service like key vault or other external loading method
  3. For development only, optionally use a secrets.py file. See Configuring secrets.py for customizing settings

Some important assumptions about the settings variable names:

  • We assume that all settings variables are CAPITALIZED. This is so it's easier to get them from the OS's environment variables.
  • Diffgram sets some defaults to most of the settings, but it is the SysAdmin responsibility to set the correct values of the settings depending on the use case of Diffgram (Dev, Sandbox, Production, Testing, etc)

Secrets are always loaded from Environment Variables
However, those can be set by another program, or loaded from another source, here are some of the ways you can load settings on Diffgram.

.env file for docker compose

In our docker compose file, we load all env variables from the .env file located at the project root. This file contains default values for a standard installation, feel free to inspect this file to see all the configurations available.

values.yaml for K8s deployments

We also offer a Helm Chart for Diffgram. You can configure the env variables by editing the values.yaml of the helm chart.

Configuring secrets.py for customizing settings (for devs/contributors)

Here is one example of loading them from a python file first before settings load. This way is usefult when doing active development of diffgram, and is recommended for developers of diffgram only and not end users.

The settings.py file contains a default import for a file named secrets.py.
This file is nonexistent in the repo and should not be tracked.

One example use case is to set the environment variables to the values depending on the DIFFGRAM_SYSTEM_MODE value or other contextual values of your server. An example of how to set the values for the settings inside the secrets.py is the following:

Example of secrets.py

if DIFFGRAM_SYSTEM_MODE in ["testing_e2e", "testing"]:
    os.environ['USER_PASSWORDS_SECRET'] = 'secret_for_test_env'
    os.environ['DB_SECRET'] = 'secret_for_test_env'
    os.environ['SECRET_KEY'] = 'secret_for_test_env'
elif DIFFGRAM_SYSTEM_MODE == 'sandbox':
    os.environ['USER_PASSWORDS_SECRET'] = 'secret_for_sanbox_env'
    
    
# Set any other variables depending on your specific use case...

Contributor Info - Environment Variables for Our Build Server (CircleCI)

Most of the configurations of our CircleCI build process are managed via environment variables set at the project level and at the context level. If you are proposing or working on improving our build system please make sure to check the values of this environment variables first and make sure to set them at the UI level inside the circleCI platform.

Secrets Management

All secrets in Diffgram are set via environment variables.

List of Secrets Used in Diffgram [Not all are used depending on the usecase]

  • USER_PASSWORDS_SECRET
  • SECRET_KEY
  • INTER_SERVICE_SECRET
  • FERNET_KEY: This one should be generated with Fernet Library
  • DIFFGRAM_AWS_ACCESS_KEY_SECRET
  • DIFFGRAM_AZURE_CONNECTION_STRING
  • SERVICE_ACCOUNT_FULL_PATH
  • MAILGUN_KEY
  • _ANALYTICS_WRITE_KEY
  • LABEL_BOX_SECRET

Defaults

We have certain default values for secrets in settings.py but they should be for local usage only and are discouraged for using in production.

Please make sure to set the secrets on environment variables and to keep strong security policies for the management and access of your production secrets, database password, access keys, etc.

Secrets Logging And Not Logging

In DIFFGRAM_SYSTEM_MODE=='production' secrets are not logged, or logged in safe way, eg bool(secret).

In development secrets may be logged.

Secrets are never sent to Diffgram Hub.

Updated 4 months ago