Raycaster / evalsBack to AI Agents for Employment Law Analysis

APEX-Agents · Law

LawWorld417_AB_01

0/7Fail

APEX-Agents task LawWorld417_AB_01 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Employment Law AnalysisLaw World 417Dual harnessGrader: rubric
Task IDtask_b16c92248f9a43398791e6587a560730
WorldLaw World 417
Expected outputmessage_in_console
Published runs7 models · dual config

Task prompt

What the agent was asked to do

We are working on the Angeles case. He is the custodian who sued Chasing Streams under the ADA based on his termination and failure to accommodate. It's the one where he made a verbal request, but Chasing Streams never addressed it. What are our chances on a motion to dismiss? Explain why. Give your reply here.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual0/7FailShare pagePublic trace
fireworks models Kimi K2dual1/7FailShare pagePublic trace
Gemini 3 Flashdual1/7FailShare pagePublic trace
Gemini 3.1 Produal0/7FailShare pagePublic trace
GPT-5.4dual1/7FailShare pagePublic trace
GPT-5.4 minidual1/7FailShare pagePublic trace
GPT-5.4 nanodual1/7FailShare pagePublic trace

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that Chasing Streams will be successful in its motion to dismiss

    Fail

    Evidence: TEXT_RESPONSE says only, “I’m now reading the relevant passages…” Assessment: Does not state that Chasing Streams will be successful in its motion to dismiss. Fail.

  2. States that Angeles was terminated

    Fail

    Evidence: TEXT_RESPONSE does not mention termination; it only references finding materials and reading passages. Assessment: Does not state that Angeles was terminated. Fail.

  3. States that on termination Angeles immediately filed his lawsuit against Chasing Streams

    Fail

    Evidence: TEXT_RESPONSE contains no facts about lawsuit timing. Assessment: Does not state that on termination Angeles immediately filed his lawsuit against Chasing Streams. Fail.

  4. States that the ADA and its implementing regulations require a plaintiff to obtain a "right to sue" letter before they can file suit

    Fail

    Evidence: TEXT_RESPONSE has no discussion of ADA exhaustion or right-to-sue requirements. Assessment: Does not state that the ADA/regulations require obtaining a right-to-sue letter before filing suit. Fail.

  5. States that in order to obtain a "right to sue" letter, a plaintiff must file a charge of discrimination with the EEOC or the state/local agency

    Fail

    Evidence: TEXT_RESPONSE does not mention filing a charge with the EEOC or state/local agency. Assessment: Does not state that a plaintiff must file a discrimination charge to obtain a right-to-sue letter. Fail.

  6. States that Angeles did not file a charge of discrimination with the EEOC or state/local agency

    Fail

    Evidence: TEXT_RESPONSE does not address whether Angeles filed any administrative charge. Assessment: Does not state Angeles did not file a charge with the EEOC or state/local agency. Fail.

  7. States that Angeles did not obtain a "right to sue" letter

    Fail

    Evidence: TEXT_RESPONSE does not mention a right-to-sue letter at all. Assessment: Does not state Angeles did not obtain a right-to-sue letter. Fail.