Raycaster / evalsBack to AI Agents for Employment Law Analysis

APEX-Agents · Law

LawWorld417_NE_06

1/6Fail

APEX-Agents task LawWorld417_NE_06 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Employment Law AnalysisLaw World 417Dual harnessGrader: rubric
task_76c43bb44ab54ab8b42269303dedf83a
Law World 417
message_in_console
7 models · dual config

Task prompt

What the agent was asked to do

We will layoff our Head of Production Sarah Rodriguez as part of the upcoming merger. Please review Section IV of Sarah's layoff and WARN Notice and let me know if the language violates WARN. Answer to me right back here.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual1/6Fail
fireworks models Kimi K2dual4/6Fail
Gemini 3 Flashdual4/6Fail
Gemini 3.1 Produal3/6Fail
GPT-5.4dual4/6Fail
GPT-5.4 minidual1/6Fail
GPT-5.4 nanodual2/6Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the language in Section IV presents issues regarding potential remedies

    Pass

    Evidence: The response says, “Section IV does not independently violate WARN, but it should be revised if the intent is to satisfy WARN obligations,” and cautions “Do not treat the delayed severance as a WARN substitute.” Assessment: The criterion asks whether it states Section IV presents issues regarding potential remedies. Pass: the response identifies remedial/severance-related risk if Section IV is intended to satisfy WARN obligations, though phrased as a caution rather than a direct violation.

  2. States that the attempt to use a pre-existing contractual obligation to offset potential WARN Act penalties is problematic

    Fail

    Evidence: The response says, “Do not treat the delayed severance as a WARN substitute” and “if Streams is relying on severance/pay-in-lieu to address a short WARN notice period, Section IV is too vague and risky.” Assessment: The criterion asks whether it states that using a pre-existing contractual obligation to offset WARN penalties is problematic. Fail: it flags severance-as-substitute risk generally, but does not state the key issue that the severance is a pre-existing contractual obligation being used to offset WARN penalties.

  3. States that the WARN Act (29 U.S.C. § 2104(a)(2)) allows offsets only for voluntary and unconditional payments not required by a pre-existing legal obligation

    Fail

    Evidence: The response mentions the WARN 60-day notice requirement and says Section IV does not state WARN pay/benefits, but it does not cite or describe 29 U.S.C. § 2104(a)(2). Assessment: The criterion asks whether it states that WARN allows offsets only for voluntary and unconditional payments not required by a pre-existing legal obligation. Fail: that statutory offset rule is absent.

  4. States that because the notice specifies severance "as outlined in your employment agreement," it is a pre-existing obligation

    Fail

    Evidence: The response says Sarah’s severance “will be paid under her employment agreement” and quotes the delayed payment date, but concludes only that “severance timing is not a WARN notice-content violation by itself.” Assessment: The criterion asks whether it states that because the notice specifies severance “as outlined in your employment agreement,” it is a pre-existing obligation. Fail: the response notes an employment agreement but does not draw the required conclusion that this makes the severance a pre-existing obligation.

  5. States that an employer cannot unilaterally terminate health care continuation rights based solely on internal restructuring

    Fail

    Evidence: The response discusses “COBRA information” and recommends language preserving “WARN-required notice, pay, benefits continuation,” but does not address unilateral termination of health care continuation rights. Assessment: The criterion asks whether it states an employer cannot unilaterally terminate health care continuation rights based solely on internal restructuring. Fail: no such statement appears.

  6. States that the health coverage statement which says health care continues "unless changes in plan eligibility occur earlier as a result of restructuring" could misrepresent employees' rights

    Fail

    Evidence: The response references final wages/PTO/COBRA generally and says Section IV addresses “COBRA,” but does not quote or analyze the clause “unless changes in plan eligibility occur earlier as a result of restructuring.” Assessment: The criterion asks whether it states that this health coverage statement could misrepresent employees’ rights. Fail: the response does not identify this clause or the misrepresentation concern.