New Engineer Welcome
Setting Up Your Local Dev Env
Assumes Already Have Basics
This assumes you already have looked at the general concepts for getting started.
There are many ways to run Diffgram locally. For the sake of development the ability to have the hot reload is important. In theory you can do this through docker too, but for the sake of this example I will show the "raw" way of running it without docker.
The Services Overview
The following services are required. Some tasks may not need all of these.
- Dispatcher.
local_dispatch.py
- Default Service (Backend)
default/main.py
- Walrus Service (Backend)
walrus/main.py
- Events handler (Backend)
eventhandlers/main.py
- Frontend
frontend
yarn run dev
Depending on your operating system, you may need to install different OS dependencies for the Python packages to work.
Each folder will have an independent requirements.py
you can decide if you place all requirements on the same python env or if you create one python env per service.
Example First Time Dev Setup
- Use the install.py script to setup your env variables. This will generate the
.env
file with the values. Optionally you can use thesecrets.py
file for local development. Check this link for more info on secrets.py - Setup local dependencies, including the database. We suggest using docker for postgres and rabbit mq. Check our Docker compose file to see examples of the services
- Run all the services
Postgres
Options
- Install locally
- Install with docker
- Use a remote postgres instance
This is done automatically as part of install.py.
To manually construct the initial database, or update to the latest version run db-init.sh
.
The structure of the database is defined in code through SQL Alchemy classes.
The migration and history of the database uses Alembic.
This will start with 77907aedd319_diffgram_initial_db
, and migrate through all revisions to present. This is a more durable approach then simply calling the SQL Alchemy classes directly.
Rabbit MQ
You can use the Docker Official Image
Default Ports - Reference Material
Defaults already set
These are the defaults in both the services and the local_dispatch.
- Dispatcher 8085 http://127.0.0.1:8085/ is the main access point
- Default 8080
- Walrus 8082
- Frontend 8081
- Postgres 5432 Set as part of database url
- Eventhandlers 8086
- RabbitMQ 5672 and 15672
Local Dependencies
- NodeJS
- Python environments. Check the requirements.txt for each service. Suggest virtual env or conda
- Export or set PYTHON_PATH
Conda example
cd replace_with_path_to_service
conda create --name diffgram_walrus python=3
pip install -r requirements.txt
Frontend Dependencies
- Install
- Build. eg to get icons
Running all The Services
1.If you are doing local development, we recommend having a shell/CLI tab per service. You can do this with any modern bash tool like ZSH.
-
For each service just execute
python main.py
and the service should start. -
For the frontend service you should run
yarn dev
to start the Node Server. -
All services assume Postgres and RabbitMQ are already up and running. Otherwise the service startup checks will fail.
-
You can access localhost:8085 on your browser and you should see the diffgram login screen. Create your first user, this user will be set up automatically as a super admin.
Updated 6 months ago