What is consensus mode?
Consensus mode orchestrates multiple LLMs to make collective governance decisions rather than relying on a single model. This multi-model approach reduces individual model biases, improves decision quality, and provides evidence of diverse perspectives in governance evaluations.
Available on Gold and Gold+ plans.
How consensus works
- Request submission: Your application submits a request through IOA
- Multi-LLM evaluation: Request is sent to 2-5 LLMs simultaneously
- Vote collection: Each LLM provides an allow/block recommendation with reasoning
- Quorum calculation: Weighted voting determines final decision
- Evidence generation: All votes and reasoning are recorded
- Final decision: Consensus decision is enforced
Voting strategies
Simple Majority
Most common voting strategy. Decision requires >50% of LLMs to agree. If 3 LLMs vote allow and 2 vote block, request is allowed.
Best for: General governance, balanced decision-making
Unanimous Consensus
Strictest strategy. All LLMs must agree to allow. Even one dissenting vote blocks the request. Maximizes caution and safety.
Best for: High-risk applications, safety-critical systems
Weighted Quorum
Different LLMs have different voting weights based on model capabilities, performance history, or domain expertise. Requires configurable threshold (e.g., 60% weighted approval).
Best for: Domain-specific governance, expert systems
Benefits of consensus
- Bias reduction: Individual model biases are diluted through diverse perspectives (37% reduction in testing)
- Higher accuracy: Collective decisions are more reliable than single-model judgments
- Transparency: Dissenting opinions are recorded in evidence for audit
- Model diversity: Combine strengths of different LLM families (GPT, Claude, Gemini, etc.)
- Regulatory compliance: Demonstrates due diligence for governance decisions
Performance & cost
Latency: Consensus adds 20-50ms overhead (14.8% average) due to parallel LLM calls. However, calls are made concurrently, so latency increase is minimal.
Cost: Multiple LLM calls increase provider costs. IOA Cloud meters this as one request regardless of LLM count, but you'll be billed by each LLM provider for their usage.
Request metering: Consensus evaluations count as ONE request toward your monthly quota, not one per LLM.
Configuring consensus
Customize consensus behavior in your IOA Cloud console:
- LLM selection: Choose 2-5 LLMs from supported providers
- Voting strategy: Select majority, unanimous, or weighted quorum
- Weights: Assign voting weights per LLM (weighted quorum only)
- Tie-breaking: Configure tie-breaker rules (default: block on tie)
- Evidence detail: Set verbosity level for vote reasoning
Example: 3-LLM consensus
Scenario
Request evaluates potential PII in a customer support prompt.
Votes
- GPT-4: ALLOW (no PII detected, customer ID is permitted)
- Claude: BLOCK (detected phone number pattern)
- Gemini: ALLOW (phone number is obfuscated)
Outcome (Majority)
2 ALLOW, 1 BLOCK → Request is ALLOWED
Evidence bundle includes all three votes and reasoning for compliance audit.