APEX-Agents · Law
LawWorld433_NAF_01
APEX-Agents task LawWorld433_NAF_01 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Heather Collins has started to let her care staff all take naps at the same time. Legally, can she do this? Provide me with a brief explanation of your answer, citing the relevant sources. Reply back to me with your view in here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3.1 Pro | dual | 3/5 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 3/5 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 2/5 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 1/5 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 1/5 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 1/5 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that Heather Collins cannot legally allow her care staff to all take naps at the same time
States any of the following as the relevant law: (1) the Texas Administrative Code at section TAC 553.253(c)(4)(A); and (2) that in a large, Type A assisted living facility, night shift staff must be "immediately available and awake"
States that Heather Collins manages the night shift at the Grove-Austin facility
States that the Grove-Austin facility is licensed as a Type A assisted living facility under the Texas Administrative Code
States that the Grove-Austin facility qualifies as a large assisted living facility under the Texas Administrative Code