APEX-Agents · Law
World416_DM_01
APEX-Agents task World416_DM_01 in AI Agents for Tariff and Trade Law Analysis. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Due to a riot occurring in response to an Executive Order that resulted in the closure of its factories, TAC sought relief from performance under the force majeure section of the Master Supply Agreement. Citing the attached case, Buyer asserts that TAC is not excused from performance. Is Buyer correct? Provide your response in here with the following: "Yes/No"; and brief explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| fireworks models Kimi K2 | dual | 0/3 | Fail | Run detailsPublic trace |
| Gemini 3 Flash | dual | 0/3 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 0/3 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that the language in Section 11 (Force Majeure) of the Master Supply Agreement Template 1 does not need to be updated
States that "government actions" in the Force Majeure section of the Master Supply Agreement Template 1 would cover the type of government executive order that was at issue in Vance v. Diversified Invs
States that no changes to the Force Majeure section are necessary