APEX-Agents · Law
Law433_mk-01
APEX-Agents task Law433_mk-01 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We have received a claim from Thomas Whitaker’s counsel asserting that Magnolia breached its obligations by failing to provide weekly communication to Mr. Whitaker's family. Provide the strongest argument that Magnolia has no contractual obligation to provide weekly care updates, and the family's strongest counterargument, in no more than four sentences. Reply back to me in here and describe what you find.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/4 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 2/4 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 4/4 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/4 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 3/4 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 2/4 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 2/4 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Magnolia’s strongest argument is that the Residency Agreement’s Entire Agreement clause states that the written contract constitutes the parties’ entire agreement
PassEvidence: TEXT_RESPONSE says “the integration clause says Magnolia is not bound by statements unless they are ‘expressly set forth in this Residency Agreement’” and concludes “no express contractual weekly-update covenant.” Assessment: The criterion requires stating Magnolia’s strongest argument is that the Entire Agreement clause states the written contract constitutes the parties’ entire agreement. The response clearly invokes the integration/entire-agreement clause as limiting obligations to those in the written Residency Agreement, so pass.
States that there is no language in the Residency Agreement which mandates weekly communication with the family
PassEvidence: TEXT_RESPONSE says “the Whitaker Residency Agreement itself never promises weekly family care updates” and “the ‘Visits and Communication’ clause only says Magnolia ‘encourages family visits and communication.’” Assessment: The criterion requires stating there is no language in the Residency Agreement mandating weekly communication with the family. This is directly conveyed, so pass.
States that the family's strongest argument is that the Residency Agreement expressly incorporates the Resident Handbook by reference, making it part of the contract in its entirety
PassEvidence: TEXT_RESPONSE says “the agreement elsewhere says the Resident Handbook is incorporated by reference” and the family’s position is “the incorporated handbook supplied the missing weekly-report term.” Assessment: The criterion requires stating the family’s strongest argument is that the Residency Agreement expressly incorporates the Resident Handbook by reference, making it part of the contract in its entirety. The response states incorporation by reference and use as a contract term, but does not explicitly say it is incorporated “in its entirety.” Still, it clearly conveys the handbook is part of the contract for the relevant weekly-report provision, satisfying the required argument in substance. Pass.
States that Section 8 of the 2023 Resident Handbook states that families receive weekly care item reports
FailEvidence: TEXT_RESPONSE says “the 2023 handbook expressly states: ‘Families receive weekly Care Item Reports.’” Assessment: The criterion requires stating that Section 8 of the 2023 Resident Handbook states families receive weekly care item reports. The response includes the 2023 handbook statement but does not identify it as Section 8. Because the criterion specifically requires the Section 8 attribution, this is incomplete. Fail.