[BUG] Consensus Can Be Calculated with Insufficient Validators

[frozenflux2]

Project

term-challenge

Description

The consensus calculation mechanism only requires a single validator's results to calculate a "consensus" score. This means if one validator completes quickly (or maliciously), they can establish the final score before other validators submit their results.

Error Message

No error - premature consensus with single validator
Final score determined by first/fastest validator

Debug Logs

// src/storage/pg.rs lines 3009-3022
// Get all evaluations for this agent
let rows = transaction
    .query(
        "SELECT score::FLOAT8, tasks_passed, tasks_total, tasks_failed, ...
         FROM validator_evaluations WHERE agent_hash = $1",
        &[&agent_hash],
    )
    .await?;

if rows.is_empty() {
    transaction.rollback().await?;
    return Err(anyhow::anyhow!("No evaluations found for agent"));
}

// Consensus calculated with ANY number of validators >= 1 !!!
// No check that rows.len() >= MIN_VALIDATORS_FOR_CONSENSUS
let count = rows.len() as f64;  // Could be 1!
let final_score = (total_score / count).clamp(0.0, 1.0);

System Information

- Bounty Version: 0.1.0
- OS: Ubuntu 24.04 LTS
- Rust: 1.75+

Screenshots

No response

Steps to Reproduce

1. Agent is assigned to 3 validators (standard)
2. Malicious validator V1 completes evaluation in 30 seconds with score 0.0
3. V1 calls log_task for all assigned tasks, triggering auto-complete
4. Auto-complete triggers consensus calculation
5. Consensus is calculated with ONLY V1's results (1 validator)
6. Final score = 0.0 (V1's manipulated score)
7. Honest validators V2 and V3 complete later - too late, consensus already stored

Expected Behavior

Consensus should require a minimum number of validators:

const MIN_VALIDATORS_FOR_CONSENSUS: usize = 2; // Or 3

let rows = transaction.query(...).await?;

if rows.len() < MIN_VALIDATORS_FOR_CONSENSUS {
    // Don't calculate consensus yet - wait for more validators
    transaction.rollback().await?;
    return Err(anyhow::anyhow!(
        "Insufficient validators for consensus: {} < {}",
        rows.len(), MIN_VALIDATORS_FOR_CONSENSUS
    ));
}

Actual Behavior

• Consensus is calculated as soon as ANY validator completes
• First validator to complete determines the score
• No minimum validator count enforced for consensus

Timeline of attack:

T+0s:   Agent assigned to V1, V2, V3
T+30s:  Malicious V1 "completes" all tasks (fake results)
T+30s:  Consensus calculated with V1's score only!
T+180s: Honest V2 completes - too late, score already finalized
T+200s: Honest V3 completes - ignored

Additional Context

Related code that DOES check validator count:

// src/storage/pg.rs lines 908-909 (leaderboard query)
HAVING COUNT(DISTINCT ve.validator_hotkey) >= 2

// src/validation/code_visibility.rs line 23
pub const MIN_VALIDATORS_FOR_VISIBILITY: usize = 3;

The leaderboard requires >= 2 validators, but consensus calculation doesn't!

Recommended fix:

1. Add MIN_VALIDATORS_FOR_CONSENSUS constant
2. Check rows.len() >= MIN_VALIDATORS_FOR_CONSENSUS before calculating
3. Return early if insufficient validators
4. Wait for more validators before finalizing score

[echobt]

Issue validated. The reported problem has been confirmed in the codebase. A Pull Request with the fix has been submitted: PlatformNetwork/term-challenge#78.