APEX-Agents · Law
World433_JS_08
APEX-Agents task World433_JS_08 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
We are in the process of doing compliance checks on the Magnolia Gardens General Resident Agreement Template. We are currently examining whether the General Resident Agreement Template is in compliance with California Law. Determine whether the General Resident Agreement Template complies with California Code of Regulations Title 22, Division 6, Chapter 8, section 87507(g)(3)(H). Write back to me with your determination in a single answer with a short explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| fireworks models Kimi K2 | dual | 3/3 | Pass | Run detailsPublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 3/3 | Pass | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that California Code of Regulations Title 22, Division 6, Chapter 8, section 87507(g)(3)(H) requires admission agreements to contain "a provision indicating that an itemized monthly statement that lists all separate charges incurred by the resident that are collected by the facility shall be provided to the resident or the resident’s representative, if any."
States that the General Resident Agreement Template does not contain a provision indicating that an itemized monthly statement that lists all separate charges incurred by the resident that are collected by the facility shall be provided to the resident or the resident’s representative
States that the General Resident Agreement Template does not comply with California Code of Regulations Title 22, Division 6, Chapter 8, section 87507(g)(3)(H)