Embed

Embed AI Supervision into your Application

📘

🔜 Preview Document

This is a preview. We expect to ship full Embedded example soon. Join our community to discuss your needs for embedded AI.

Introduction

You may already be thinking of ways to embed annotation into your application. This document will help surface some of the reasons why that's the right idea, and how Diffgram can help.

Your users are often the best people to provide supervision (annotation). Your users already have the context of what they want. Your users can provide supervision "for free", which scales much better than trying to hire ever larger central annotation teams.

As a single engineer, you can drop Diffgram Embedded into your application, and your share of the work is nearly "complete". As a team, you can skip organizing or hiring a huge annotation department. Instead you can engage your users.

When to use Embedded

Embedded works best for ongoing maintenance, domains where a "super user" can configure items for other users, and domains where the "long tail" of complexity is high.

Area	Type
Initial model training	Central team
Maintenance	Embedded
User preferences	Embedded

Benefits

1. Cost effective at scale vs only central team
2. Efficient, users already have context
3. Automated - users can create their own supervision on demand.
4. End users have vastly greater social incentives to get it right.
   And with no 3rd party (central team), it opens novel use cases, especially with sensitive or proprietary data.
5. Adds options for "user style" and user preferences.

How it works

1. Add small amount of code to your frontend. React Embedded Sample
2. Setup Diffgram (Setup Guide)
3. Start collecting end user supervision.

Managing User Expectations

Supervision that resemble regular forms (multiple select, radial, free text, attributes) etc. are broadly applicable to most computer users. More complex interfaces (such as image annotation, video, 3d, etc) will limit your audience.

Keep in mind that you can have supervision goals different from the media type. So for example your predictions may include polygons, but then your supervision may be just attributes.

If you are building internal tools, we encourage you to have training sessions and engage staff directly. If your end users are outside your company you can include messaging in your application. Over time the concept of human supervision will become more widely accepted.

Automation

You can integrate Embedded with Workflows to create automatic training and retraining paths. This creates a system wide automation from end user to ML.

You can also integrate other automation approaches, for example users can make corrections to existing predictions.

Embedded also makes some approaches less relevant. When you have so many more people annotating there is much more throughput, and the significance is more in the user’s knowledge rather than exactly how fast they input it.

How does Embedded compare to Customization?

The Embedded interface can be customized, but no matter how you customize the look and feel, users still have to visit a portal to do the annotation. Even if the portal is “linked” deeply, this is usually a less optimal user experience. By embedding it, you can maintain your user experience.

Why use Diffgram vs rolling your own?

To roll your own you need to build out the frontend visuals that are shown to the user, figure out the behind the scenes data loading, security etc. Then you need to create a database to store that and so much more. Pretty soon you are recreating Diffgram! Diffgram makes it way faster to get started, and to scale your efforts.

History

Embedded is an evolution. Early tools were built for a single user. Then multi-user tools came, but they centered around a “Portal”. Now embed allows you to distribute, or decentralize your annotation supervision.

We have been working towards this direction for a while, and recently have started to create the specific technical implementations on the front end to make this possible.

Code Samples

React Embedded Sample

Getting started

1. Explore code sample
2. Install Diffgram through NPM
3. Start building

Related Reading

See Development System
and the SDK