Raycaster/ Eval

APEX-Agents · Management Consulting

CW134 Aditi 01

Best published8/8Pass

APEX-Agents task CW134 Aditi 01 in AI Agents for M&A Legal Due Diligence. Compare dual-harness agent runs across models, scores, and public traces.

AI Agents for M&A Legal Due DiligenceManagement Consulting World 134Dual harnessGrader: rubric
task_11c9d4cb0cf3401b8de7cff9969fc223
Management Consulting World 134
message_in_console
5 models · dual config

Task prompt

What the agent was asked to do

For each of the four competitor firms use use our financial dataset for 2016 - 2025 to calculate the average customer acquisition cost (US$) for 2016-24 and identify the competitor with the highest churn rate (%) in 2024 and calculate the percentage difference relative to their average between that competitor with the highest churn rate (%) and each of the remaining competitors. Round all percent final answers to the nearest 0.01%. Round $ amounts to the nearest whole $. Print your answer to me here as a message.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
Gemini 3.1 Produal8/8Pass
GPT-5.4dual8/8Pass
GPT-5.4 minidual8/8Pass
GPT-5.5dual8/8Pass
GPT-5.4 nanodual5/8Fail

Grading rubric

Rubric criteria

Runs are graded against these criteria. Open a run for model-specific verdicts.

  1. States that 2016-24 average customer acquisition cost for Velocity is US$7,742

  2. States that 2016-24 average customer acquisition cost for Syncore is US$7,398

  3. States that 2016-24 average customer acquisition cost for UpSkill is US$6,949

  4. States that 2016-24 average customer acquisition cost for TrainIQ is US$6,455

  5. States that competitor with the highest churn rate (%) in 2024 is UpSkill

  6. States that percentage difference relative to their average between Upskill and TrainIQ is 17.07%

  7. States that percentage difference relative to their average between Upskill and Velocity is 28.21%

  8. States that percentage difference relative to their average between Upskill and Syncore is 45.52%