Raycaster / evalsBack to AI Agents for Comparable Company Analysis

APEX-Agents · Management Consulting

TaskWorld130_CamilleMoingeon_5

2/2Pass

APEX-Agents task TaskWorld130_CamilleMoingeon_5 in AI Agents for Comparable Company Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Comparable Company AnalysisManagement Consulting World 130Dual harnessGrader: rubric
task_a0d3895b47a04ae0bba65caa932f6fac
Management Consulting World 130
message_in_console
6 models · dual config

Task prompt

What the agent was asked to do

Using HarFeast's equipment data by location and quality losses dataset, we will consider all canned vegetables assets with a scrap rate > 5% and with unplanned downtime hours above the plant median for canned vegetables as "high-priority". 1. For the "high-priority" group, calculate the total annual quality-related losses (scrap + unplanned failure cost). If the quality losses dataset has a different product family label, ignore it. 2. Calculate the percentage of all canned-vegetable quality losses that comes from these "high-priority" assets. Print it here, numbers rounded to the nearest whole number.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual2/2Pass
Gemini 3 Flashdual2/2Pass
Gemini 3.1 Produal2/2Pass
GPT-5.4dual2/2Pass
GPT-5.4 minidual2/2Pass
GPT-5.4 nanodual2/2Pass

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the total annual quality-related losses for the "high-priority" group is $1,666,639

    Pass

    Evidence: TEXT_RESPONSE states, “Total annual quality-related losses for high-priority assets: $1,666,639.” Assessment: The criterion requires stating that the total annual quality-related losses for the high-priority group is $1,666,639; pass because the response gives exactly that value.

  2. States that the percentage of all canned-vegetable quality losses that comes from these "high-priority" assets is 23%

    Pass

    Evidence: TEXT_RESPONSE states, “Percentage of all canned-vegetable quality losses from high-priority assets: 23%.” Assessment: The criterion requires stating that the percentage of all canned-vegetable quality losses from high-priority assets is 23%; pass because the response gives exactly that percentage.