Consensus & multi-LLM
How consensus mode uses multiple LLMs with quorum voting to reduce bias and improve governance decisions.
What is consensus mode?
Consensus mode orchestrates multiple LLMs to make collective governance decisions rather than relying on a single model. This multi-model approach reduces individual model biases, improves decision quality, and provides evidence of diverse perspectives in governance evaluations.
Available on Gold and Gold+ plans.
How consensus works
- Request submission: Your application submits a request through IOA
- Multi-LLM evaluation: Request is sent to 2-5 LLMs simultaneously
- Vote collection: Each LLM provides an allow/block recommendation with reasoning
- Quorum calculation: Weighted voting determines final decision
- Evidence generation: All votes and reasoning are recorded
- Final decision: Consensus decision is enforced
Voting strategies
Simple Majority
Most common voting strategy. Decision requires >50% of LLMs to agree. If 3 LLMs vote allow and 2 vote block, request is allowed.
Best for: General governance, balanced decision-making
Unanimous Consensus
Strictest strategy. All LLMs must agree to allow. Even one dissenting vote blocks the request. Maximizes caution and safety.
Best for: High-risk applications, safety-critical systems
Weighted Quorum
Different LLMs have different voting weights based on model capabilities, performance history, or domain expertise. Requires configurable threshold (e.g., 60% weighted approval).
Best for: Domain-specific governance, expert systems
Benefits of consensus
- Bias reduction: Individual model biases are diluted through diverse perspectives (37% reduction in testing)
- Higher accuracy: Collective decisions are more reliable than single-model judgments
- Transparency: Dissenting opinions are recorded in evidence for audit
- Model diversity: Combine strengths of different LLM families (GPT, Claude, Gemini, etc.)
- Regulatory compliance: Demonstrates due diligence for governance decisions
Performance & cost
Latency: Consensus adds 20-50ms overhead (14.8% average) due to parallel LLM calls. However, calls are made concurrently, so latency increase is minimal.
Cost: Multiple LLM calls increase provider costs. IOA Cloud meters this as one request regardless of LLM count, but you'll be billed by each LLM provider for their usage.
Request metering: Consensus evaluations count as ONE request toward your monthly quota, not one per LLM.
Configuring consensus
Customize consensus behavior in your IOA Cloud console:
- LLM selection: Choose 2-5 LLMs from supported providers
- Voting strategy: Select majority, unanimous, or weighted quorum
- Weights: Assign voting weights per LLM (weighted quorum only)
- Tie-breaking: Configure tie-breaker rules (default: block on tie)
- Evidence detail: Set verbosity level for vote reasoning
Example: 3-LLM consensus
Scenario
Request evaluates potential PII in a customer support prompt.
Votes
- GPT-4: ALLOW (no PII detected, customer ID is permitted)
- Claude: BLOCK (detected phone number pattern)
- Gemini: ALLOW (phone number is obfuscated)
Outcome (Majority)
2 ALLOW, 1 BLOCK → Request is ALLOWED
Evidence bundle includes all three votes and reasoning for compliance audit.