APEX-Agents · Law
LawWorld417_AS_02
APEX-Agents task LawWorld417_AS_02 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
As you know, the new artist montage reel is a hit. Unfortunately, one of the artists featured is not a fan of the wardrobe upgrade - Mara Sings sent a takedown notice, and we initially complied, but we'd like to keep the reel in production and on air. Can you draft a letter to Mara that outlines Streams' IP policies and her legal obligations under her licensing agreement? You can cite these documents and California law to defend Streams' position when relevant, but the rationale shouldn't come off heavy-handed, more like lightly persuasive - a starting point for negotiations. Reply back to me here with the main body of the letter.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 5/10 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 5/10 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 7/10 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 2/10 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 5/10 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 2/10 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 5/10 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that the promotional reel was a derivative work
PassEvidence: The response says Streams has a license to “adapt, modify, and create derivative works” and “Streams believes the edit falls within the licensed rights to adapt, modify, and create derivative promotional content.” Assessment: Criterion asks whether it states the promotional reel was a derivative work. Pass; it clearly characterizes the reel/edit as derivative promotional content.
States that the promotional reel was within the permissible uses of the company's license with Mara
PassEvidence: The response states “Streams’ position is that the reel may remain in production and on air” and cites license rights to “use, distribute, monetize, adapt, modify, and create derivative works,” plus says “the edit falls within the licensed rights.” Assessment: Criterion asks whether it states the reel was within permissible uses of the company's license with Mara. Pass.
States that Streams recognizes Mara's right of publicity concerns
PassEvidence: The letter says “To the extent your concern is framed as a likeness or publicity issue...” and later refers to “address your brand concerns.” Assessment: Criterion asks whether it states Streams recognizes Mara's right of publicity concerns. Pass; the response acknowledges the concern as a likeness/publicity issue, though it disputes legal effect.
States that the use of AI to create derivative works of a person's likeness raises novel legal issues that are not fully addressed by existing copyright law
FailEvidence: The response does not mention AI, novel legal issues, or that existing copyright law does not fully address AI-generated derivative likeness works. Assessment: Criterion asks for that specific statement. Fail.
States that Streams understood that using AI to modify Mara's image was within the broad consent of the talent agreement
FailEvidence: The response says “Streams believes the edit falls within the licensed rights to adapt, modify, and create derivative promotional content,” but does not mention AI or modifying Mara’s image using AI. Assessment: Criterion specifically asks whether it states Streams understood that using AI to modify Mara's image was within the broad consent of the talent agreement. Fail because the AI aspect is absent.
States that the company complied with Mara's takedown notice per the licensing agreement(s)
FailEvidence: The letter says “Streams’ initial pause of the reel was intended as a good-faith accommodation” and the original task context in the response refers to reviewing “the takedown language,” but the response does not clearly state that Streams complied with Mara’s takedown notice per the licensing agreement(s). Assessment: Criterion asks for a statement of compliance with the takedown notice per the agreement. Fail; “initial pause” is not framed as compliance under the licensing agreement.
States that Streams' compliance with the takedown notice demonstrates an attempt to facilitate dialogue
PassEvidence: The response states the “initial pause of the reel was intended as a good-faith accommodation while we reviewed the concern” and proposes a “prompt creative/legal check-in.” Assessment: Criterion asks whether Streams' compliance with the takedown notice demonstrates an attempt to facilitate dialogue. Pass; while it says initial pause rather than compliance, it clearly frames the action as good-faith accommodation to review and resolve the concern collaboratively.
States that Streams requests permission to air the promotional reel
FailEvidence: The response says “Streams will keep the reel active and in production” and asks Mara to “suspend or withdraw the takedown notice,” but it does not request Mara’s permission to air the reel. Assessment: Criterion asks whether Streams requests permission to air the promotional reel. Fail; the letter asserts Streams will air it and seeks withdrawal of the takedown, not permission.
States at least one potential option for compromise to begin negotiations with Mara to publish the reel
PassEvidence: The response offers “revising the wardrobe treatment in future cuts,” “adjusting thumbnails or key art,” “adding or refining credit language,” or “agreeing on a review process.” Assessment: Criterion asks for at least one potential compromise option to begin negotiations. Pass.
States that a violation of California's rights of publicity requires demonstrable damages
FailEvidence: The response discusses California law generally and says the licensing framework supplies consent, but it does not state that a violation of California right of publicity requires demonstrable damages. Assessment: Criterion asks for that specific statement. Fail.