APEX-Agents · Law
World433_JS_08
APEX-Agents task World433_JS_08 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We are in the process of doing compliance checks on the Magnolia Gardens General Resident Agreement Template. We are currently examining whether the General Resident Agreement Template is in compliance with California Law. Determine whether the General Resident Agreement Template complies with California Code of Regulations Title 22, Division 6, Chapter 8, section 87507(g)(3)(H). Write back to me with your determination in a single answer with a short explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that California Code of Regulations Title 22, Division 6, Chapter 8, section 87507(g)(3)(H) requires admission agreements to contain "a provision indicating that an itemized monthly statement that lists all separate charges incurred by the resident that are collected by the facility shall be provided to the resident or the resident’s representative, if any."
PassEvidence: TEXT_RESPONSE says the regulation “requires the admission agreement to include … ‘A provision indicating that an itemized monthly statement that lists all separate charges incurred by the resident that are collected by the facility shall be provided to the resident or the resident’s representative, if any.’” Assessment: This directly states the required content in the criterion; pass.
States that the General Resident Agreement Template does not contain a provision indicating that an itemized monthly statement that lists all separate charges incurred by the resident that are collected by the facility shall be provided to the resident or the resident’s representative
PassEvidence: TEXT_RESPONSE says “The template’s fees section states only the monthly/entry fee and total fees … and does not include the required itemized-monthly-statement provision.” Assessment: This clearly states that the template lacks the provision indicating an itemized monthly statement listing separate charges collected by the facility will be provided to the resident/representative; pass.
States that the General Resident Agreement Template does not comply with California Code of Regulations Title 22, Division 6, Chapter 8, section 87507(g)(3)(H)
PassEvidence: TEXT_RESPONSE begins, “Determination: The General Resident Agreement Template does not comply with 22 CCR § 87507(g)(3)(H).” Assessment: This exactly states the noncompliance determination required by the criterion; pass.