Raycaster / evalsBack to AI Agents for Employment Law Analysis

APEX-Agents · Law

LawWorld417_NE_03

2/6Fail

APEX-Agents task LawWorld417_NE_03 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Employment Law AnalysisLaw World 417Dual harnessGrader: rubric
task_384b35183b374b5482edfff4f071c834
Law World 417
message_in_console
6 models · dual config

Task prompt

What the agent was asked to do

We're helping one of ChasingStreams's talent partners create a project for distribution on their channel(s). In addition to providing the production stages, ChasingStreams will contribute $10,000 cash plus $20,000 in employee time. We're otherwise staying hands-off with the project. Do we need any separate agreements with the talent? Give m the answer straight here and explain why, briefly.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual2/6Fail
fireworks models Kimi K2dual0/6Fail
Gemini 3.1 Produal6/6Pass
GPT-5.4dual5/6Fail
GPT-5.4 minidual3/6Fail
GPT-5.4 nanodual1/6Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States Yes, a separate revenue agreement between ChasingStreams and each of the talent is required for the project

    Pass

    Evidence: The response says, “Yes — use a short project-specific agreement or addendum with the talent” and “get a narrow project addendum/SOW with the participating talent.” Assessment: The criterion requires stating that a separate revenue agreement between ChasingStreams and each talent is required. The response says yes to a separate project agreement/addendum with the talent, but does not specifically identify it as a “revenue agreement” or say “each” talent. Still, it clearly advises a separate agreement with participating talent for the project economics. Pass.

  2. States that the IP & Content Ownership Policy requires a separate revenue agreement for a Co-Created Work

    Fail

    Evidence: The response references “existing talent paperwork” and “IP diligence materials,” but does not mention the “IP & Content Ownership Policy” or that it “requires a separate revenue agreement for a Co-Created Work.” Assessment: The criterion specifically requires this policy-based statement. It is absent. Fail.

  3. States that the project is a Co-Created Work under the IP & Content Ownership Policy

    Fail

    Evidence: The response describes “this new arrangement adds project-specific economics and resources,” but never uses or paraphrases the classification “Co-Created Work” under the IP & Content Ownership Policy. Assessment: The criterion requires stating the project is a Co-Created Work. The response does not. Fail.

  4. States that ChasingStreams contributed $30,000 in production costs to the Talent's project

    Pass

    Evidence: The response states the project includes “$10,000 cash, and $20,000 of employee time” and earlier notes “ChasingStreams will contribute $10,000 cash plus $20,000 in employee time” by implication. Assessment: The criterion requires stating ChasingStreams contributed $30,000 in production costs. The response gives the two components totaling $30,000 but does not characterize them as “$30,000 in production costs.” This is close enough in substance because it states the full contribution amounts. Pass.

  5. States that ChasingStreams must contribute at least $25,000 to the cost of a production for it to constitute a Co-Created Work

    Fail

    Evidence: The response does not mention any $25,000 threshold or minimum contribution requirement. Assessment: The criterion requires stating that ChasingStreams must contribute at least $25,000 to production cost for Co-Created Work status. Absent. Fail.

  6. States that ChasingStream's financial contribution to the project's production cost exceeds the threshold amount required for a project to constitute a Co-Created Work under the IP & Content Ownership Policy

    Fail

    Evidence: The response states the contribution components but does not mention any threshold amount or say the contribution exceeds it under the IP & Content Ownership Policy. Assessment: The criterion requires stating that ChasingStreams’s financial contribution exceeds the threshold for Co-Created Work status. Absent. Fail.