Labeling Standard Operating Procedure (SOP)

Your labels power AI models. Accuracy matters — both for the quality of the data and for your trust score and earnings.

How Labeling Works

1. You receive a task — each task presents data (an image, text, or other media) and asks you to apply a label from a predefined set of options.
2. You submit your label — choose the option that best describes the data.
3. Consensus is checked — your label is compared against labels from other workers on the same task.
4. You get paid — if your label matches the consensus, you earn BSV.

The Consensus Model

ASG uses a multi-worker consensus model:

• Each task is assigned to multiple workers (typically 3)
• A task reaches consensus when a majority of workers agree (e.g., 2 out of 3)
• Your label is evaluated against the consensus, not against a single "correct" answer

This means:

• If you and at least one other worker agree → consensus reached → you get paid
• If all three workers disagree → no consensus → the task may be re-queued

info

Consensus protects both data quality and workers. No single person decides what's "correct" — the group does.

Quality Standards

Do

• Read the full task instruction before labeling — don't assume based on the preview
• Take your time on unfamiliar content — speed matters less than accuracy
• Be consistent — apply the same standard across similar items
• Skip tasks you're unsure about rather than guessing (when skip is available)

Don't

• Don't rush — random or careless labels damage your trust score
• Don't label content you can't see or understand (e.g., blurry images, languages you don't speak)
• Don't coordinate with other workers on specific labels — this undermines the independence that makes consensus meaningful
• Don't use automation or scripts to submit labels — this violates the Code of Conduct and will result in a permanent ban

Label Categories

Tasks specify which labels are valid. Common patterns include:

Task Type	Example Labels	What to Look For
Image Classification	cat, dog, bird, other	Primary subject of the image
Sentiment Analysis	positive, negative, neutral	Overall tone of the text
Content Moderation	safe, unsafe, borderline	Whether content meets guidelines
Entity Recognition	person, place, organization	What the highlighted text refers to

The specific labels and instructions vary by task — always read the task-level instructions provided with each item.

Handling Edge Cases

When the correct label isn't obvious:

1. Re-read the task instruction — the answer is often in the details
2. Look for the dominant characteristic — if an image shows a cat and a dog, label based on what the instruction asks (e.g., "primary subject")
3. When genuinely ambiguous, choose the label that most people would reasonably pick — consensus rewards agreement, not perfection
4. If a task seems broken (missing image, garbled text), skip it if possible or label it as "other" / the closest neutral option

warning

Never submit the same label for every task. Pattern-detection systems flag uniform labeling as potential bot behavior, which damages your trust score.

Your Label History

You can review your past labels in the History tab of the worker app. Each entry shows:

• The task ID
• Your submitted label
• The task status (completed, pending consensus)
• When you submitted it

Use your history to identify patterns in your accuracy and improve over time.