Measure Data Quality (Annotations & More)

Quality Assurance Features

Traffic Control: Speed up Workflow with Automatic Review Routing

All tasks have an optional review. As an admin you set this review frequency between 0 and 100% of the tasks. The moment each task is complete the reviewer is able to access the task. This is per task, so work can flow freely. The reviewer can pass or reject the task. Optionally they can leave a general comment, or a more specific issue on a specific annotation. The results of this review ratio can be further analyzed eg "Who always is getting them wrong".

QA Slideshow: Reduce Costly Errors

Fully integrated Quality Control Slideshow helps reviews view each annotation one at a time.

Reduce Context Switching Costs with Discussions & Issue Tracking

Fully integrated issue tracking.
See Issues

Get New Team Members Certified with Training and Exams

You can access an Annotation capability, without risking real work, through the Exams practice function

Each time an Annotator goes to start an Exam, it's a fresh copy. If you are familiar with Video Game instances it's similar to that concept.
Afterwards there is a process to grade and optionally award credentials.

These credentials can be optionally required on future tasks.
Basically it allows you to enforce the exam requirement - so a user can't start working on a job till the exam that's relevant to it is complete.
For the grading process, for a few data types this can be nearly automatic, but for many this will require a human review.

Diffgram is one of the only options I'm aware of that offers this level of depth of integrated Exams.

Hold People Accountable with Per User Reporting

See annotations, time spent, and more, all per user.

Hold Teams Accountable with Per Task Reporting and more

Easily see the status of a given batch of work.

Stages Workflows - Advanced Data & Review Pipelines

This lets you set different steps, etc. This is a super flexible workflow concept.

Customizable Automations

Use Interactive Annotation Automation to easily create customized quality assurance scripts.

Customizable Alerts and Automations

Get email notifications as tasks progress or in the event of exceptions.
Customize and setup webhooks.
Setting Up Webhooks
Setting Up Email Notifications

Reduce Human Errors with Human Centered Tasks

Automatic Task Distribution

To manage workflow at scale it takes a team. And that team has distinct roles, like admin and annotator. Diffgram has persona focused user experiences optimized for every member of your team.

Annotate. Resume. Browse

More Effective Curation of Data

Quickly identify what data to label and reduce costs labeling data that doesn't improve the model.

Easily query data after the fact to find errors.

Skip downloading and unzipping massive datasets. Explore data instantly through the browser.
Visualize in seconds multiple datasets. Compare models without extra computation.

(More automatic surfacing features coming soon)

Feature Requests

Please feel free to create a new issue

Updated over 2 years ago