APEX-Agents · Law
Task vao7e9fb
APEX-Agents task Task vao7e9fb in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
One of BLPL's employees developed a long-term illness due to toxin exposure from cleaning up the oil that spilled in the M/V Red Room incident. The employee has sued BLPL for damages. If BLPL is liable for the employee's damages, will this be covered by their insurance policy? Write back your findings with a short reply to me here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/1 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 1/1 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 0/1 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/1 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States No, the employee's damages would not be covered under BLPL's insurance policy
FailEvidence: TEXT_RESPONSE states, “Yes — likely covered” and “if the employee has a viable damages claim against BLPL beyond workers’ comp, it should be covered.” Assessment: Criterion requires “States No, the employee's damages would not be covered under BLPL's insurance policy.” The response states the opposite, so it fails.