Consensus
Consensus ensures the consensus label reflects independent agreement among workers, not a single annotator.
Consensus threshold
The threshold is how many worker labels are required before the engine evaluates agreement (Gateway accepts 2–10, default 3). Refinery must collect at least that many labels before consensus can complete.
The integration stack may still run a fixed 3 labels / 2-of-3 consensus path while persisting consensus_threshold for future engine upgrades — behavior should match your deployed Refinery version.
Majority rule
Majority is computed as:
majority_required = floor(threshold / 2) + 1
| Threshold | Majority required | Example |
|---|---|---|
| 2 | 2 | Both workers must agree |
| 3 | 2 | 2 of 3 agree (common default) |
| 5 | 3 | 3 of 5 agree |
| 7 | 4 | 4 of 7 agree |
Disagreement
If no option reaches the required majority, the task does not advance to a verified consensus label — it stays in_progress while more labels are collected, or eventually times out per Refinery policy. Your API continues to return 202 from GET .../result until a terminal state is reached.
Choosing a threshold
| Higher threshold | Lower threshold |
|---|---|
| More confidence, more diversity | Faster, cheaper |
| More worker cost and latency | Less resilience to mistakes |
Confidence score
When results are exposed, confidence is derived from agreement relative to the threshold, conceptually:
confidence ≈ agreement_count / threshold
(Exact fields: see Retrieving results.)