Use Diffgram with Your Apps

Introduction

Do you want to integrate and use AI in your applications? Or create dynamically generated applications? Diffgram helps you get there with a "Systems Level" approach. Use cases:

• Pushing labeling work to your end users, reducing AI maintenance costs, and limiting PII data
• Gaining more flexibility to update and ship new AI products faster. Creating systems to generate AI concepts "on demand".
• Reducing bespoke one-off systems and leaning into programmatic system approaches.

Diffgram forms a key part of your apps by providing a single place to put, organize, and access multiple Schemas, Predictions and Blobs.

Push Labeling Work to Your Users

By pushing labeling work to your end users you will reduce your PII exposure and reduce maintenance costs. Key aspects of Diffgram that can help you achieve this:

• Use programmatic approaches to create and access objects, especially Query Syntax Reference.
• Use Customization options to make the look and feel match your end users context, including White Label UI Customization Introduction and Custom Buttons.
• Use OIDC Config (OAuth 2.0) to enable end user login experiences.

By combining programmatic integrations, customizations, and OAuth, you can create a custom experiences. You can further extend this by exploring Workflow.

Next we will discuss what this can look like in practice with an End to End example.

End to End Example

How to engineer programmatic labeling experiences, unique to each of your users, that translate into ML apps.

The following is a baseline process that gives your app the ability to translate incoming user or customer specific data (with their own Schema) into tasks completed by that user. And the ability for your application to update in real time based on changing data from your users.

Steps:

1. Setup OIDC Config (OAuth 2.0) to permission your users access across apps. Programmatically create Diffgram users as needed.
2. Programmatically create a dataset name with a variable key relevant to your apps structure. New Dataset new().
3. Programmatically ingest the data (e.g. media files) to that newly created Dataset Compound Files Ingestion.
4. Programmatically create a new job using that dataset (Job New SDK). Reference a (previously created) customer specific Schema for example by storing a mapping of your user keys to Schema IDs or using a known naming method for the Schema. ( Schemas List) See notes below for more details on this step. Optionally reference custom UI Schema, as noted below.
5. Programmatically Get the Task IDs (for that Job) to show to the end user by calling Task List or Next Task.
6. In your application, create a regular HTML link to diffgram.your_domain.com/task/{id}.
7. The user can then click that link, login with OAuth, and complete the task.
8. Optionally use Custom Buttons to direct the user back to your application. Optionally use White Label UI Customization. For example to display specific messages to users to help them gain more context or limit the tools they can use.
9. Set Up Webhooks and then Query Data to get the latest slice, using that data in your own pipelines. Conceptually we assume here that this freshly produced training data is the key ingredient and the consumption process by ML systems is mostly or completely automated.
10. Optionally extend the ability to make innovative tailored applications by capturing and collecting changing customer needs through your own app and then call APIs like Attribute Template Update to update options.

Notes

• The above steps are meant to be illustrative and are not exhaustive.
• Schema Notes
  • The Schema itself may still be created manually by your team. Or the Schema can be created programmatically too.
  • Create entire New Schemas based on your application logic
  • Capture and collect changing customer needs through your own application and then call APIs like Attribute Template Update to update options.
• The key steps of this process stay the same even if:
  • Each of your users data resides on different clouds. See New Connection.
  • Each of your users have a different Schema.
  • The Schema changes.
• Some aspects may be manually inspected, updated, or created in the Diffgram UI by your team.
• If you have already been using the Diffgram APIs and SDKs you may already be familiar with some of these concepts. Note that some require differences to be effective in an integrated, programmatic context, vs using a scripting or pre-defined context. For example:
  • Naming, keys, IDs, and storing of those references
  • Error handling
  • Assumptions around getting lists, IDs, etc. vs consuming through Task List UI pages
  • Validation and assumptions around creating and modifying Schemas
  • User creation and flow concepts
  • Use of UI customization options
• In this context, users should need less or little training in order to be effective in terms of completing tasks.
• Due to the UI nature of Diffgram, as needed you can inspect the in progress work, use the Reporting features, etc.
• You can extend functions by Permalink - Deep Linking in more depth, for example linking to a specific Attribute that should open, a specific video frame, a specific annotation, or a specific issue.
• Optionally further extend these concepts by using Diffgram Workflow.

Embedded UI is Optional

Concepts like the white-label UI customization and OAuth seamless login make it unnecessary to embed the frontend UI. However, in Preview, we are exploring options here, see Embed. We recommend getting a baseline set up, e.g. centered around task IDs and links, up and running first before exploring embedded options.

More Examples

More zoomed-in examples.

Gaining more flexibility to update on demand

Update the Schema through your custom UI

• Capture and collect changing customer needs through your own application and then call APIs like Attribute Template Update to update options.
• Create entire New Schemas based on your application logic

Reduce bespoke system usage

Diffgram makes it easier to test, validate, and swap out multiple ML processes. This reduces the burden of custom logic for specific cases.

• Setting Up Webhooks and then Query Data to get the latest slice, using that data in your own pipelines.
• Use Workflow

Connect users to your existing outputs

Display data to end users

• Generate queries in your app and show views of the Data Explorer to show multiple files at once
• Display prediction results for specific files by linking to the specific file ID

Create Deeper Experiences with Workflow (Preview)

Workflow is an optional advanced step that provides deeper experiences to create even more advanced applications on demand. In the above context, we are essentially limited to rendering very specific pre-defined experiences and traditional software engineering integrations. Workflow provides a "no code" way to link together more complex training data needs.

Workflow is like a bridge to provide that last missing piece between the functionality in diffgram and the functionality in the outside world. In theory anything is an API call, but in practice the level of support matters greatly. Workflow provides an optional way to organize, manage, and visualize the training data side.

Anything you can do in another more general purpose workflow system we encourage that. Workflow is not trying to replace or compete with any of those things. Only things that need to be in Workflow to support training data work should be there. View a preview of Workflows here.

Summary

Diffgram provides the key system concepts to build programmatic labeling experiences for your users. This article covered some high level concepts, a specific end to end example, and some optional context for more advanced uses.