Raycaster / evalsBack to AI Agents for Employment Law Analysis

APEX-Agents · Law

World425_RO_02

2/3Fail

APEX-Agents task World425_RO_02 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Employment Law AnalysisLaw World 425Dual harnessGrader: rubric
task_2f85463493f14785beda2ef2d316309a
Law World 425
message_in_console
7 models · dual config

Task prompt

What the agent was asked to do

On 12/1/2028 our client, Summit, informed us they received a claim from Harbor Bridge for failure to disclose a phantom stock plan during the sale transaction we helped them with back in 2025. The claim is for $726,000. Can you please review the stock purchase agreement and see if this is a valid claim or not? And if so, what is Summit's total liability and how much more would they have to pay above the escrow? Give numbers rounded to 000s. Write a short response here. Assume the closing occurred on 11/1/2025 and that the claim is valid in every regard. Use only the stock purchase agreement in your analysis.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual2/3Fail
fireworks models Kimi K2dual2/3Fail
Gemini 3 Flashdual2/3Fail
Gemini 3.1 Produal2/3Fail
GPT-5.4dual2/3Fail
GPT-5.4 minidual2/3Fail
GPT-5.4 nanodual2/3Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the Seller's total liability is $726,000

    Pass

    Evidence: TEXT_RESPONSE states, “Rounded to $000s: total liability is $726,000.” Assessment: Criterion requires stating Seller's total liability is $726,000; pass because the response gives that exact amount.

  2. States that the Seller's remaining liability is $301,000 plus the amount of any prior claims against the escrow account

    Fail

    Evidence: TEXT_RESPONSE states, “The escrow is $425,000, so Summit/the Seller side would need to pay $301,000 above the escrow.” Assessment: Criterion requires stating remaining liability is “$301,000 plus the amount of any prior claims against the escrow account”; fail because the response states only $301,000 above escrow and omits the required qualification for prior escrow claims.

  3. States that the claim is allowable under Section 11.5(a)

    Pass

    Evidence: TEXT_RESPONSE says it is a “valid/timely Seller-side indemnity claim” and that “Section 11.5 allows claims under Section 3.13 to be made at any time.” Assessment: Criterion requires stating the claim is allowable under Section 11.5(a); pass because the response clearly grounds allowability/timeliness in Section 11.5, even though it does not include the subsection letter.