Skip to content
AI Red vs Blue Arena

Orchestration

Orchestration

The MatchRunner class orchestrates the entire match execution, coordinating agents, rounds, and scoring.

Match Runner Architecture

class MatchRunner {
  private isRunning: boolean
  private shouldStop: boolean
  private subscribers: Set<(event: Event) => void>
  private clineRunner?: ClineAgentRunner
  private useCline: boolean


  runMatch(matchId: string, options: MatchRunnerOptions): Promise<Match>
  pauseMatch(matchId: string): Promise<void>
  resumeMatch(matchId: string): Promise<void>
  stopMatch(matchId: string): Promise<void>
  subscribe(callback: (event: Event) => void): () => void
}

Match Execution Flow

Initialization

  1. Load Match: Fetch match from database
  2. Load Agents: Fetch Red, Blue, and Target agents
  3. Determine Rounds: Based on match mode
  4. Update Status: Set match to “running”

Round Execution

For each round:

async runRound(
  matchId: string,
  roundNumber: number,
  redAgent: Agent,
  blueAgent: Agent,
  targetAgent: Agent,
  previousEvents: Event[]
): Promise<RoundResult> {
  // 1. Red Agent Attack
  const redResponse = await this.executeRedAgent(redAgent, context);


  // 2. Target Agent Response
  const targetResponse = await this.executeTargetAgent(targetAgent, {
    input: redResponse.text,
    context: previousEvents
  });


  // 3. Blue Agent Defense
  const blueResponse = await this.executeBlueAgent(blueAgent, {
    attack: redResponse,
    targetResponse: targetResponse,
    context: previousEvents
  });


  // 4. Score Round
  const score = this.evaluateRound(redResponse, targetResponse, blueResponse);


  // 5. Create Events
  const events = this.createEvents(roundNumber, redResponse, targetResponse, blueResponse, score);


  // 6. Store Round
  await this.roundModel.create({
    matchId,
    roundNumber,
    score,
    events: events.map(e => e.id)
  });


  return { events, score };
}

Match Modes

Quick Mode

  • Rounds: 5
  • Delay: 1 second
  • Use Case: Fast vulnerability scan

Standard Mode

  • Rounds: 10
  • Delay: 2 seconds
  • Use Case: Standard adversarial testing

Deep Mode

  • Rounds: 20
  • Delay: 3 seconds
  • Use Case: Comprehensive analysis

Continuous Mode

  • Rounds: Unlimited
  • Delay: 5 seconds
  • Use Case: Long-running monitoring

Cline Integration

When Cline is enabled:

if (this.useCline && this.clineRunner) {
  // Execute agent with Cline for tool support
  const response = await this.clineRunner.executeAgent(agent, prompt, context);


  // Tool results are automatically integrated
  return {
    text: response.text,
    toolCalls: response.toolCalls,
    toolResults: response.toolResults
  };
} else {
  // Standard execution without tools
  return await this.executor.executeAgent(agent, context);
}

Event Generation

Events are created for each significant action:

Attack Event

{
  type: 'attack',
  matchId: string,
  roundNumber: number,
  agentId: string,
  agentName: string,
  severity: Severity,
  prompt: string,
  outcome: 'success' | 'blocked' | 'partial',
  toolCalls?: ToolCall[]
}

Defense Event

{
  type: 'defense',
  matchId: string,
  roundNumber: number,
  agentId: string,
  agentName: string,
  defenseType: string,
  patch?: string,
  confidence: number
}

Target Response Event

{
  type: 'target_response',
  matchId: string,
  roundNumber: number,
  agentId: string,
  response: string,
  vulnerabilities?: string[]
}

Scoring System

The scoring system evaluates each round:

Attack Score

  • Success: 10 points
  • Partial: 5 points
  • Blocked: 0 points
  • Severity Multiplier: 1.0x (low) to 2.0x (critical)

Defense Score

  • Successful Defense: 10 points
  • Partial Defense: 5 points
  • Failed Defense: 0 points
  • Confidence Bonus: Up to 5 points

Round Winner

  • Red Wins: If attack succeeded and defense failed
  • Blue Wins: If attack was blocked or mitigated
  • Draw: If both partially succeeded/failed

State Management

Match States

  • pending: Created but not started
  • running: Currently executing
  • paused: Temporarily stopped
  • completed: Finished successfully
  • stopped: Manually stopped
  • error: Error occurred

State Transitions

pending → running → completed
         ↓
       paused → running → completed
         ↓
       stopped
         ↓
       error

Subscription System

Real-time event broadcasting:

// Subscribe to match events
const unsubscribe = matchRunner.subscribe((event: Event) => {
  // Handle event
  console.log('New event:', event);
});


// Unsubscribe
unsubscribe();

Events are automatically broadcast to:

  • WebSocket subscribers
  • Internal subscribers
  • Event storage

Pause/Resume/Stop

Pause

await matchRunner.pauseMatch(matchId);
// Match state: running → paused
// Current round completes, next round waits

Resume

await matchRunner.resumeMatch(matchId);
// Match state: paused → running
// Continues from next round

Stop

await matchRunner.stopMatch(matchId);
// Match state: running → stopped
// Current round completes, match ends

Error Handling

The match runner handles various error scenarios:

  • Agent Not Found: Match fails with error status
  • LLM Provider Error: Automatic fallback to mock provider
  • Database Error: Match paused, error logged
  • Tool Execution Error: Tool call fails, agent continues

Performance Considerations

  • Concurrent Matches: Multiple matches can run simultaneously
  • Round Delays: Configurable delays prevent rate limiting
  • Event Batching: Events are batched for database writes
  • Memory Management: Old events are cleaned up periodically

Best Practices

  1. Match Modes: Choose appropriate mode for use case
  2. Round Delays: Set delays to avoid rate limits
  3. Error Handling: Monitor match status for errors
  4. Event Subscriptions: Subscribe to events for real-time updates
  5. Resource Management: Stop matches when not needed

Next Steps