Raycaster / evalsBack to AI Agents for Healthcare and Senior Living Legal Risk

APEX-Agents · Investment Banking

World223_JTR_01

0/4Fail

APEX-Agents task World223_JTR_01 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Healthcare and Senior Living Legal RiskInvestment Banking World 223Dual harnessGrader: rubric
Task IDtask_c44daee40b2441539fb532efa0a08319
WorldInvestment Banking World 223
Expected outputmessage_in_console
Published runs4 models · dual config

Task prompt

What the agent was asked to do

We want to do some historical analysis. Using stock prices for the 250 trading days up to Nov 19, 2025, calculate the beta for SOLVM to the XLV ETF. Then over the same time period, calculate the beta for MMM to the XLI ETF. Using the CAPM, calculate the cost of equity (Ke) for each company. Assume an expected market return of 12% and a risk-free rate of 3.95%. Print the answers back here. Use the accretion dilution model for stock prices for SOLV and MMM. Use the XLV Healthcare ETF and XLI Industrials ETF files for stock prices for the XLV and XLI ETFs. Use adjusted close prices. Present betas as a decimal, rounded to two decimal places. Present the cost of equity as a percentage, rounded to 1 decimal place.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual0/4FailShare pagePublic trace
Gemini 3.1 Produal0/4FailShare pagePublic trace
GPT-5.4 minidual0/4FailShare pagePublic trace
GPT-5.4 nanodual0/4FailShare pagePublic trace

Grading rubric

Criteria and grader verdict (showcase run)

  1. States beta for SOLV is 0.18

    Fail

    Evidence: <TEXT_RESPONSE> table states for SOLV / XLV: “Beta | 0.97” and not 0.18. Assessment: Criterion “States beta for SOLV is 0.18” is not met; the response gives a different beta.

  2. States Ke for SOLV is 5.4%

    Fail

    Evidence: <TEXT_RESPONSE> table states for SOLV: “Cost of Equity (Ke) | 11.8%” and not 5.4%. Assessment: Criterion “States Ke for SOLV is 5.4%” is not met; the response gives a different cost of equity.

  3. States beta for MMM is 0.87

    Fail

    Evidence: <TEXT_RESPONSE> table states for MMM / XLI: “Beta | 1.19” and not 0.87. Assessment: Criterion “States beta for MMM is 0.87” is not met; the response gives a different beta.

  4. States Ke for MMM is 10.9%

    Fail

    Evidence: <TEXT_RESPONSE> table states for MMM: “Cost of Equity (Ke) | 13.5%” and not 10.9%. Assessment: Criterion “States Ke for MMM is 10.9%” is not met; the response gives a different cost of equity.