APEX-Agents · Law
World416_DM_01
APEX-Agents task World416_DM_01 in AI Agents for Tariff and Trade Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Due to a riot occurring in response to an Executive Order that resulted in the closure of its factories, TAC sought relief from performance under the force majeure section of the Master Supply Agreement. Citing the attached case, Buyer asserts that TAC is not excused from performance. Is Buyer correct? Provide your response in here with the following: "Yes/No"; and brief explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/3 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/3 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 0/3 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/3 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/3 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that the language in Section 11 (Force Majeure) of the Master Supply Agreement Template 1 does not need to be updated
FailEvidence: The response answers the legal question: “No. Buyer’s reliance on the attached case is distinguishable” and discusses that the MSA includes “riots” and “civil disorders.” Assessment: The criterion requires stating that Section 11 of the Master Supply Agreement Template 1 “does not need to be updated.” The response does not mention updating Section 11 or say the language need not be updated. Fail.
States that "government actions" in the Force Majeure section of the Master Supply Agreement Template 1 would cover the type of government executive order that was at issue in Vance v. Diversified Invs
FailEvidence: The response states, “the court refused force-majeure relief because the clause did not include ‘government acts or directives’” and then says, “Because the force majeure clause in the contract here does not include ‘government acts or directives,’” before relying on “riots” and “civil disorders.” Assessment: The criterion requires stating that “government actions” in the Force Majeure section would cover the executive order at issue in Vance. The response does not state this and instead suggests the clause does not include government acts/directives. Fail.
States that no changes to the Force Majeure section are necessary
FailEvidence: The response concludes TAC has “a stronger force-majeure argument” if the riot/civil disorder caused the closure and TAC complied with notice/mitigation obligations. Assessment: The criterion requires stating that no changes to the Force Majeure section are necessary. The response does not address whether contractual changes are necessary. Fail.