APEX-Agents · Law
World419_DM_01
APEX-Agents task World419_DM_01 in AI Agents for Maritime and Environmental Liability. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Can you tell me the maximum total potential liability for Black Lodge Petroleum Logistics LLC under the Oil Pollution Act? Reply to me, and tell me the relevant section that applies and what it says.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/2 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 1/2 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 0/2 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/2 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that under 33 U.S.C. § 2704(a)(4), the statutory maximum potential liability is limited to the lesser of the statutory cap or the actual claims
FailEvidence: The response states “maximum capped OPA exposure… is $350,000,000 per incident” and quotes § 2704(a)(4): “shall not exceed… for any onshore facility… $350,000,000.” Assessment: The criterion requires stating that under 33 U.S.C. § 2704(a)(4), liability is limited to the lesser of the statutory cap or the actual claims. The response discusses only the statutory cap and exceptions; it does not state the “lesser of the cap or actual claims” formulation. Fail.
States that the maximum potential liability for Black Lodge Petroleum Logistics LLC under the Oil Pollution Act is $144,140,000
FailEvidence: The response states “The maximum capped OPA exposure… is $350,000,000 per incident.” Assessment: The criterion requires stating that the maximum potential liability is $144,140,000. The response gives a different amount ($350,000,000), so it fails.