DocumentationAPI ReferencePlaygroundCommunity

New Engineer Welcome

Suggest Edits

Setting Up Your Local Dev Env

📘

Assumes Already Have Basics

This assumes you already have looked at the general concepts for getting started.

There are many ways to run Diffgram locally. For the sake of development the ability to have the hot reload is important. In theory you can do this through docker too, but for the sake of this example I will show the "raw" way of running it without docker.

The Services Overview

The following services are required. Some tasks may not need all of these.

  1. Dispatcher. local_dispatch.py
  2. Default Service (Backend) default/main.py
  3. Walrus Service (Backend) walrus/main.py
  4. Events handler (Backend) eventhandlers/main.py
  5. Frontend frontend yarn run dev

Depending on your operating system, you may need to install different OS dependencies for the Python packages to work.

Each folder will have an independent requirements.py you can decide if you place all requirements on the same python env or if you create one python env per service.

Example First Time Dev Setup

  1. Use the install.py script to setup your env variables. This will generate the .env file with the values. Optionally you can use the secrets.py file for local development. Check this link for more info on secrets.py
  2. Setup local dependencies, including the database. We suggest using docker for postgres and rabbit mq. Check our Docker compose file to see examples of the services
  3. Run all the services

Postgres

Options

  1. Install locally
  2. Install with docker
  3. Use a remote postgres instance

This is done automatically as part of install.py.

To manually construct the initial database, or update to the latest version run db-init.sh.

The structure of the database is defined in code through SQL Alchemy classes.
The migration and history of the database uses Alembic.

This will start with 77907aedd319_diffgram_initial_db, and migrate through all revisions to present. This is a more durable approach then simply calling the SQL Alchemy classes directly.

Rabbit MQ

You can use the Docker Official Image

Default Ports - Reference Material

📘

Defaults already set

These are the defaults in both the services and the local_dispatch.

  1. Dispatcher 8085 http://127.0.0.1:8085/ is the main access point
  2. Default 8080
  3. Walrus 8082
  4. Frontend 8081
  5. Postgres 5432 Set as part of database url
  6. Eventhandlers 8086
  7. RabbitMQ 5672 and 15672

Local Dependencies

  1. NodeJS
  2. Python environments. Check the requirements.txt for each service. Suggest virtual env or conda
  3. Export or set PYTHON_PATH

Conda example

cd replace_with_path_to_service
conda create --name diffgram_walrus python=3
pip install -r requirements.txt

Frontend Dependencies

  1. Install
  2. Build. eg to get icons

Running all The Services

1.If you are doing local development, we recommend having a shell/CLI tab per service. You can do this with any modern bash tool like ZSH.

  1. For each service just execute python main.py and the service should start.

  2. For the frontend service you should run yarn dev to start the Node Server.

  3. All services assume Postgres and RabbitMQ are already up and running. Otherwise the service startup checks will fail.

  4. You can access localhost:8085 on your browser and you should see the diffgram login screen. Create your first user, this user will be set up automatically as a super admin.

Updated 6 months ago