Review Loop 101

Data Labeling Review Loop

Process

  1. In Progress
  2. In Review
  3. If Fail Review, go back to in progress.
  4. If Pass Review, complete.
24372437

How to enable reviews?

During task creation, select "Yes" to enable reviews.
Select a percent or check "Review All".

29382938

Tasks Start In Progress

910910

Then Move to Review

Once the initial annotator marks the files as ready for review it goes to the review stage.

935935

Review Prompt

The reviewer approves or rejects. They may optionally leave a comment.

981981

Rejection moves back to In progress

Rejecting it now shows as "Requires Changes". Re completing the task will send it for review again. This also makes the tasks eligible for automatic assignment from "next task" buttons.

10801080

Approval Moves to Complete

Approving marks the task as completed. This will also trigger any actions to move it to another stage of work.

Creating Multiple Review Stages

Create a two stage pipeline.

Keywords

Review
Review loop
Quality Assurance