Raycaster / evalsBack to AI Agents for Cross-Border Regulatory Review

APEX-Agents · Management Consulting

World 128_RG_02

4/4Pass

APEX-Agents task World 128_RG_02 in AI Agents for Cross-Border Regulatory Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Cross-Border Regulatory ReviewManagement Consulting World 128Dual harnessGrader: rubric
task_cb687edb826643a7857ab1f4cde2c0a5
Management Consulting World 128
message_in_console
5 models · dual config

Task prompt

What the agent was asked to do

Calculate the change in the chances of success for each of the frontier business units. Use the formula 0.25 × the sum of all operational constraints. The numerical values of the operational constraints are available in the attached file. Please provide your answer right here. Round all final answers to 2 decimal places, i.e., 0.01%

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual4/4Pass
Gemini 3.1 Produal4/4Pass
GPT-5.4dual4/4Pass
GPT-5.4 minidual4/4Pass
GPT-5.4 nanodual4/4Pass

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the absolute change in the chances of success for AmensaDrive is 0.00%

    Pass

    Evidence: <TEXT_RESPONSE> table lists AmensaDrive with “Change in chances of success | 0.00%”. Assessment: Criterion asks whether it states the absolute change for AmensaDrive is 0.00%; pass because the response clearly states that value.

  2. States that the absolute change in the chances of success for Skylink Atmos is 1.00%

    Pass

    Evidence: <TEXT_RESPONSE> table lists “SkyLink Atmos” with “Change in chances of success | 1.00%”. Assessment: Criterion asks whether it states the absolute change for Skylink Atmos is 1.00%; pass despite capitalization difference because the required unit and value are clearly conveyed.

  3. States that the absolute change in the chances of success for AmensaMech is 0.75%

    Pass

    Evidence: <TEXT_RESPONSE> table lists AmensaMech with “Change in chances of success | 0.75%”. Assessment: Criterion asks whether it states the absolute change for AmensaMech is 0.75%; pass because the response clearly states that value.

  4. States that the absolute change in the chances of success for SolisOne is 3.25%

    Pass

    Evidence: <TEXT_RESPONSE> table lists SolisOne with “Change in chances of success | 3.25%”. Assessment: Criterion asks whether it states the absolute change for SolisOne is 3.25%; pass because the response clearly states that value.