How to get your local set up and running for engineering.
- Read this doc in full
- Set up Must Have Dependencies
- Running the Services
- Use the Main Entry port of 8085
- Test the installation is working
- Further reading: Backend Architecture
Using docker-compose (Install) provides a complete local playground with minimal manual effort. Whereas setting up a bare-metal installation is maximum effort. In general, the reasons to do the Bare-Metal approach are:
- Full control of the system, including changing dependencies, hot reload, etc.
- No dependency on docker.
Depending on your needs, you may wish to run some portion of the system in docker.
Please note that you must have these dependencies set up correctly.
- Code Access
- OS Env Set Up
- Python Environments.
- Diffgram Env Variables
- BLOB Storage
- First Database Migration
- Rabbit MQ
Clone the repository.
Each service has a few assumptions around the OS. Things like which python version, etc. Use the
Dockerfiles as a reference. These files describe the specifics that are known to work, such as which python env, sub libraries etc.
You may run some parts of the system in Docker directly.
Each service has an independent
The python path must be set. The specifics of setting this vary by OS. Please search for the specifics for your machine.
For each service:
- Create a virtual env using Conda,
venv, or your preferred provider.
pip install -r requirements.txtto install the python dependencies.
conda create --name diffgram_walrus python=3
pip install -r requirements.txt
You may need to install different OS dependencies for certain Python packages to work. Those are outlined in the OS Env step above.
Separate from OS and Python setup, there is a configuration file for Diffgram itself.
To create this
.env file use the install.py script to set up your env variables. This will generate the
.env file with the values. The file should be placed in your diffgram directory. To learn more about general concepts see Secrets.
Also see Domain Configuration - Some of these settings (e.g. Env variable section) may be needed even when the domain is local.
You must configure a storage provider, e.g. Minio, AWS, GCP, etc
You must have a Postgres DB setup.
- Install locally
- Install with docker
- Use a remote postgres instance
- To manually construct the initial database, or update to the latest version run
- Manually call
alembic upgrade head. Note the python path must be set or this may fail.
The structure of the database is defined in code through SQL Alchemy classes. The migration and history of the database uses Alembic. This will start with
77907aedd319_diffgram_initial_db, and migrate through all revisions to present. This is a more durable approach then simply calling the SQL Alchemy classes directly.
Follow the official install guide.
NodeJS must be installed.
- Install NodeJS
yarn installto install local front end packages.
- Build. eg to get icons
Once you have the dependences fully installed you can run the services.
Please note there is a
local_dispatch.py file that acts as an
Ingress between the services and is the main entry to the system.
It's important to use the dispatcher (default port
8085). to ensure the overall system works as expected. Otherwise there will be unexpected and out-of-band errors. Do not try to access through the frontend service directly.
If you are doing local development, one option is to have a shell/CLI tab per service. You can do this with any modern bash tool like ZSH or BAT scripts in Windows.
- Activate the virtual env.
- For each backend service execute the
python main.pyto start the service. Load flask with hot reload on.
For the frontend service you should run
yarn dev to start the Node Server.
Start Storage (e.g MinIO) if applicable. For example in Windows
start "" minio.exe server e:/minio
It is Assumed that Postgres and RabbitMQ are already up and running (e.g. at System startup). Otherwise the service startup checks will fail.
You can access localhost:8085 on your browser and you should see the diffgram login screen.
Create your first user, this user will be set up automatically as a super admin.
These are the default ports in both the services and the local_dispatch.
|Main Entry, Emulate Production Ingress
yarn run dev
|Set as Database URL
Updated 3 months ago