Raycaster / evalsBack to AI Agents for Financial Statement Analysis

APEX-Agents · Management Consulting

World 133 EL Task 03

1/1Pass

APEX-Agents task World 133 EL Task 03 in AI Agents for Financial Statement Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Financial Statement AnalysisManagement Consulting World 133Dual harnessGrader: rubric
task_d01a20f9321341b39059a7629a365232
Management Consulting World 133
message_in_console
5 models · dual config

Task prompt

What the agent was asked to do

Estimate the contribution (in $M) that Summit's loyalty program members will make to its 2025E Program EBITDA in $ millions. Assume that Summit's loyalty program members will contribute, as a % of 2025E Program EBITDA, an amount equal to the member-weighted average % of stays by members in the most recent financial year across Hilton, Hyatt, and Marriott. Use our profitability workbook, the 10Ks for Hilton and Hyatt, and Marriot's annual report for 2024. - If data is unavailable for any of the three competitors listed, discard that competitor in your member-weighted average % of stays by members calculation. - If a range is stated instead of an exact numerical member count (e.g. we have 'over 10 million members') use the headline value ('10 million'). Write your answers out here. Report all numerical values to 1 decimal place.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual1/1Pass
Gemini 3.1 Produal1/1Pass
GPT-5.4dual1/1Pass
GPT-5.4 minidual1/1Pass
GPT-5.4 nanodual1/1Pass

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the estimated contribution made to 2025E Program EBITDA by Summit's loyalty members is $170.7M

    Pass

    Evidence: TEXT_RESPONSE states, “Estimated Summit loyalty-member contribution to 2025E Program EBITDA: $170.7M” and concludes “Summit’s loyalty program members are estimated to contribute **$170.7M**.” Assessment: The criterion asks whether it states the estimated contribution is $170.7M; pass.