APEX-Agents · Law
LawWorld433_NAF_02
APEX-Agents task LawWorld433_NAF_02 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
The attorney for Thomas Whitaker wants to submit this matter for arbitration in Texas. Reply to me in here, drafting an output explaining whether that would be consistent with Mr. Stoker's Litigation Plan and the Residency Agreement for Thomas Whitaker.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3.1 Pro | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 2/3 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 2/3 | Fail | Run detailsPublic trace |
| Gemini 3 Flash | dual | 1/3 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 1/3 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 0/3 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 0/3 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that going to arbitration with the Whitakers is consistent with Mr. Stoker's Litigation Plan
States that the Residency Agreement requires mandatory arbitration in Los Angeles, California
States that participating in arbitration in Texas is not consistent with the Residency Agreement