Review Loop 101

Data Labeling Review Loop

Process

  1. In Progress
  2. In Review
  3. If Fail Review, go back to in progress.
  4. If Pass Review, complete.

How to enable reviews?

During task creation, select "Yes" to enable reviews.
Select a percent or check "Review All".

Tasks Start In Progress

Then Move to Review

Once the initial annotator marks the files as ready for review it goes to the review stage.

Review Prompt

The reviewer approves or rejects. They may optionally leave a comment.

Rejection moves back to In progress

Rejecting it now shows as "Requires Changes". Re completing the task will send it for review again. This also makes the tasks eligible for automatic assignment from "next task" buttons.

Approval Moves to Complete

Approving marks the task as completed. This will also trigger any actions to move it to another stage of work.

Creating Multiple Review Stages

Create a two stage pipeline.

Keywords

Review
Review loop
Quality Assurance


Did this page help you?