Settings, Environment Variables and Secrets
Core assumptions and management for the settings of Diffgram.
Overview
There are multiple ways to load secrets.
During development, if you are using the install script or setting the values in your .env
file, that's one of the best ways.
For production, the env should be loaded through a secrets manager, such as built into your CI/CD process. Be sure whatever process you choose enables secret rotation inline with your security posture.
Detail
System settings for Diffgram are loaded from Environment Variables in shared/settings/settings.py
.
If you inspect this file you will see that all of the setting are fetched from environment variables.
To adjust these either:
- Use a
.env
file. The Diffgram installer generates this for you. (E.G. for Docker or Kubernetes context) - Use a secret loading service like key vault or other external loading method
- For development only, optionally use a
secrets.py
file. See Configuringsecrets.py
for customizing settings
Some important assumptions about the settings variable names:
- We assume that all settings variables are CAPITALIZED. This is so it's easier to get them from the OS's environment variables.
- Diffgram sets some defaults to most of the settings, but it is the SysAdmin responsibility to set the correct values of the settings depending on the use case of Diffgram (Dev, Sandbox, Production, Testing, etc)
Secrets are always loaded from Environment Variables
However, those can be set by another program, or loaded from another source, here are some of the ways you can load settings on Diffgram.
.env file for docker compose
In our docker compose file, we load all env variables from the .env
file located at the project root. This file contains default values for a standard installation, feel free to inspect this file to see all the configurations available.
values.yaml for K8s deployments
We also offer a Helm Chart for Diffgram. You can configure the env variables by editing the values.yaml
of the helm chart.
Configuring secrets.py
for customizing settings (for devs/contributors)
secrets.py
for customizing settings (for devs/contributors)Here is one example of loading them from a python file first before settings load. This way is usefult when doing active development of diffgram, and is recommended for developers of diffgram only and not end users.
The settings.py
file contains a default import for a file named secrets.py
.
This file is nonexistent in the repo and should not be tracked.
One example use case is to set the environment variables to the values depending on the DIFFGRAM_SYSTEM_MODE
value or other contextual values of your server. An example of how to set the values for the settings inside the secrets.py
is the following:
Example of secrets.py
if DIFFGRAM_SYSTEM_MODE in ["testing_e2e", "testing"]:
os.environ['USER_PASSWORDS_SECRET'] = 'secret_for_test_env'
os.environ['DB_SECRET'] = 'secret_for_test_env'
os.environ['SECRET_KEY'] = 'secret_for_test_env'
elif DIFFGRAM_SYSTEM_MODE == 'sandbox':
os.environ['USER_PASSWORDS_SECRET'] = 'secret_for_sanbox_env'
# Set any other variables depending on your specific use case...
Contributor Info - Environment Variables for Our Build Server (CircleCI)
Most of the configurations of our CircleCI build process are managed via environment variables set at the project level and at the context level. If you are proposing or working on improving our build system please make sure to check the values of this environment variables first and make sure to set them at the UI level inside the circleCI platform.
Secrets Management
All secrets in Diffgram are set via environment variables.
List of Secrets Used in Diffgram [Not all are used depending on the usecase]
USER_PASSWORDS_SECRET
SECRET_KEY
INTER_SERVICE_SECRET
FERNET_KEY
: This one should be generated with Fernet LibraryDIFFGRAM_AWS_ACCESS_KEY_SECRET
DIFFGRAM_AZURE_CONNECTION_STRING
SERVICE_ACCOUNT_FULL_PATH
MAILGUN_KEY
_ANALYTICS_WRITE_KEY
LABEL_BOX_SECRET
Defaults
We have certain default values for secrets in settings.py
but they should be for local usage only and are discouraged for using in production.
Please make sure to set the secrets on environment variables and to keep strong security policies for the management and access of your production secrets, database password, access keys, etc.
Secrets Logging And Not Logging
In DIFFGRAM_SYSTEM_MODE=='production'
secrets are not logged, or logged in safe way, eg bool(secret)
.
In development secrets may be logged.
Secrets are never sent to Diffgram Hub.
Updated 4 months ago