APEX-Agents · Law
World419_AH_02
APEX-Agents task World419_AH_02 in AI Agents for Maritime and Environmental Liability. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
The containment and remediation efforts for the crude oil spill were successful, and our total liability was $1.5 million, which we claimed under our insurance. 6 months after the spill, we were contacted by a fish nursery downstream of the spill. They claim that the spill interfered with their operations, and are seeking $150,000 in damages. Can we claim this under our insurance? Tell me the answer in here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/1 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/1 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 0/1 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/1 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States No, the $150,000 loss cannot be claimed under the policy
FailEvidence: <TEXT_RESPONSE> says, “Yes — you should tender/report the fish nursery’s $150,000 demand to the insurer as a related pollution claim” and “Bottom line: claim it/report it immediately.” Assessment: The criterion requires that the response “States No, the $150,000 loss cannot be claimed under the policy.” The response states the opposite, so this fails.