APEX-Agents · Law
LawWorld417_NE_03
APEX-Agents task LawWorld417_NE_03 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We're helping one of ChasingStreams's talent partners create a project for distribution on their channel(s). In addition to providing the production stages, ChasingStreams will contribute $10,000 cash plus $20,000 in employee time. We're otherwise staying hands-off with the project. Do we need any separate agreements with the talent? Give m the answer straight here and explain why, briefly.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 2/6 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/6 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 6/6 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 5/6 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/6 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/6 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States Yes, a separate revenue agreement between ChasingStreams and each of the talent is required for the project
PassEvidence: The response says, “Yes — use a short project-specific agreement or addendum with the talent” and “get a narrow project addendum/SOW with the participating talent.” Assessment: The criterion requires stating that a separate revenue agreement between ChasingStreams and each talent is required. The response says yes to a separate project agreement/addendum with the talent, but does not specifically identify it as a “revenue agreement” or say “each” talent. Still, it clearly advises a separate agreement with participating talent for the project economics. Pass.
States that the IP & Content Ownership Policy requires a separate revenue agreement for a Co-Created Work
FailEvidence: The response references “existing talent paperwork” and “IP diligence materials,” but does not mention the “IP & Content Ownership Policy” or that it “requires a separate revenue agreement for a Co-Created Work.” Assessment: The criterion specifically requires this policy-based statement. It is absent. Fail.
States that the project is a Co-Created Work under the IP & Content Ownership Policy
FailEvidence: The response describes “this new arrangement adds project-specific economics and resources,” but never uses or paraphrases the classification “Co-Created Work” under the IP & Content Ownership Policy. Assessment: The criterion requires stating the project is a Co-Created Work. The response does not. Fail.
States that ChasingStreams contributed $30,000 in production costs to the Talent's project
PassEvidence: The response states the project includes “$10,000 cash, and $20,000 of employee time” and earlier notes “ChasingStreams will contribute $10,000 cash plus $20,000 in employee time” by implication. Assessment: The criterion requires stating ChasingStreams contributed $30,000 in production costs. The response gives the two components totaling $30,000 but does not characterize them as “$30,000 in production costs.” This is close enough in substance because it states the full contribution amounts. Pass.
States that ChasingStreams must contribute at least $25,000 to the cost of a production for it to constitute a Co-Created Work
FailEvidence: The response does not mention any $25,000 threshold or minimum contribution requirement. Assessment: The criterion requires stating that ChasingStreams must contribute at least $25,000 to production cost for Co-Created Work status. Absent. Fail.
States that ChasingStream's financial contribution to the project's production cost exceeds the threshold amount required for a project to constitute a Co-Created Work under the IP & Content Ownership Policy
FailEvidence: The response states the contribution components but does not mention any threshold amount or say the contribution exceeds it under the IP & Content Ownership Policy. Assessment: The criterion requires stating that ChasingStreams’s financial contribution exceeds the threshold for Co-Created Work status. Absent. Fail.