Review Loop 101
Data Labeling Review Loop
Process
- In Progress
- In Review
- If Fail Review, go back to in progress.
- If Pass Review, complete.
How to enable reviews?
During task creation, select "Yes" to enable reviews.
Select a percent or check "Review All".
Tasks Start In Progress
Then Move to Review
Once the initial annotator marks the files as ready for review it goes to the review stage.
Review Prompt
The reviewer approves or rejects. They may optionally leave a comment.
Rejection moves back to In progress
Rejecting it now shows as "Requires Changes". Re completing the task will send it for review again. This also makes the tasks eligible for automatic assignment from "next task" buttons.
Approval Moves to Complete
Approving marks the task as completed. This will also trigger any actions to move it to another stage of work.
Creating Multiple Review Stages
Create a two stage pipeline.
Keywords
Review
Review loop
Quality Assurance
Updated almost 3 years ago