Update Production

Helm Production Updates

Make sure you have the latest helm chart version by doing

git clone https://github.com/diffgram/diffgram-helm

Then change the diffgramVersion on the values.yaml file to the new version released. And redeploy chart with updated values.

Breaking Changes

Breaking changes tracking.

DB Upgrades / Deadlock Prevention

We use Alembic to manage upgrades of the database schema. Some of these upgrade may include new indexes and constraints. We strongly suggest that you reduce the database access to a minimum to prevent deadlocks on an index creation as it can cause unexpected CPU spikes and degrade the entire system's performance.

Helm Timeouts

When using helm upgrade you may need to add the --timeout flag if the index or migration to add will require significant time due to the large amount of data to index. You can upgrade with the timeout flag in the following manner:

helm upgrade diffgram . --timeout 5000

🚧

Index Upgrades

Use a HA (High Availability) database system for uptime during maintenance, or run during a low usage period or schedule a dedicated maintenance window to prevent deadlocks when upgrading DB schema.

Production Overview

This is usually automated, e.g. through a github action

  1. Initialize tracking on your preferred deployment tracking pipeline
  2. Migrate the database using the provided almebic migrations
  3. Pull the latest images
  4. Run tests
  5. Migrate traffic

Updating the SDK

pip install diffgram --upgrade

Warnings

❗️

Use Extreme Caution When Updating Kubernetes Cluster Version

Read more