Raycaster/ Eval

APEX-Agents · Management Consulting

World125_Task01_NA

Best published4/4Pass

APEX-Agents task World125_Task01_NA in AI Agents for Cross-Border Regulatory Review. Compare dual-harness agent runs across models, scores, and public traces.

AI Agents for Cross-Border Regulatory ReviewManagement Consulting World 112.1Dual harnessGrader: rubric
task_dbf1b00ea3cc4a59af744f370f7a1acf
Management Consulting World 112.1
message_in_console
5 models · dual config

Task prompt

What the agent was asked to do

Using the 2024 annual report, identify whether Impact Therapeutics sells primarily branded drugs. A company sells primarily branded drugs if the majority of drugs have launch dates within the last 7 years. Please use the BCG report to determine the average COGS as a % of revenue for competitors in Impact Therapeutics' segment, based on whether they sell primarily generic drugs, branded, or a mix. Round to the nearest %. In both the global and US geos, state whether Impact Therapeutics is above the segment average calculated from the data in the BCG report. Write your response as a reply to me here.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
Gemini 3.1 Produal4/4Pass
GPT-5.4dual4/4Pass
GPT-5.4 minidual4/4Pass
GPT-5.4 nanodual4/4Pass
GPT-5.5dual4/4Pass

Grading rubric

Rubric criteria

Runs are graded against these criteria. Open a run for model-specific verdicts.

  1. States that Impact Therapeutic is a company that primarily sells branded drugs

  2. States the average of COGS as a share of revenue for companies that primarily sell branded drugs is 23%

  3. States that the global COGS as a share of revenue for Impact Therapeutics in 2024 is higher than the average of 23%

  4. States that the US COGS as a share of revenue for Impact Therapeutics in 2024 is higher than the average of 23%