APEX-Agents · Law
Law433_mk_02
APEX-Agents task Law433_mk_02 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Can you edit the existing Grove Residency Agreement for Thomas Whitaker to make a revised residency agreement template that we can use for new residents. Please also update/add to the last sentence of the paragraph J in section IV of the agreement based on the 2025 Grove internal operations manual. Thanks.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.4 | dual | 6/7 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 5/7 | Fail | Run detailsPublic trace |
| Gemini 3 Flash | dual | 5/7 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 5/7 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 5/7 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 4/7 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/7 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
Removes any reference to "Thomas Whitaker"
Removes any reference to "Unit Number 111"
Removes any reference to the date August 1, 2023
States all of the following: (1) that the target resident-to-staff ratio is 12:1 and (2) that temporary changes in the staff-to-resident ratio of up to 15:1 is permitted
States all of the following: (1) that individualized care plan assessments are performed every six months and (2) that individualized care plan assessments are performed as staffing allows
States all of the following individuals may count to the overall coverage ratios if assisting with non-medical support tasks in the revised residency agreement: (i) volunteers at the residency, (ii) interns or other such individuals at the residency through an internship, apprenticeship or other such program, and (iii) family visitors of a resident
States that residents categorized as “Stable” may defer reassessment of their individualized care plan for up to twelve (12) months with administrative approval