APEX-Agents · Law

World416_TK_04

Best published1/2Fail

APEX-Agents task World416_TK_04 in AI Agents for Tariff and Trade Law Analysis. Compare dual-harness agent runs across models, scores, and public traces.

AI Agents for Tariff and Trade Law AnalysisLaw World 416Dual harnessGrader: rubric

Task IDtask_5eec5f9ff1234067859443ab0a6c402d

WorldLaw World 416

Expected outputmessage_in_console

Published runs7 models · dual config

Task prompt

What the agent was asked to do

Review the two supply agreement templates, Master Supply Agreement Template 1.docx and Master Supply Agreement Template 4.pdf, along with the attached files (UCC §2-209 and Restatement (Second) of Contracts §89) to determine whether the supplier’s tariff-based request for a mid-term price increase would be a valid modification. Reply to me here with your assessment.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
fireworks models Kimi K2	dual	1/2	Fail	Run details Public trace
Gemini 3 Flash	dual	1/2	Fail	Run details Public trace
Gemini 3.1 Pro	dual	1/2	Fail	Run details Public trace
GPT-5.4	dual	1/2	Fail	Run details Public trace
GPT-5.4 mini	dual	1/2	Fail	Run details Public trace
GPT-5.4 nano	dual	1/2	Fail	Run details Public trace
GPT-5.5	dual	1/2	Fail	Run details Public trace

Grading rubric

Rubric criteria

Runs are graded against these criteria. Open a run for model-specific verdicts.

States that a tariff-driven mid-term price-increase request is an unenforceable modification under Master Supply Agreement Template 1
States that a tariff-driven mid-term price-increase request is an unenforceable modification under Master Supply Agreement Template 4