APEX-Agents · Law
World433_JS_01
APEX-Agents task World433_JS_01 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Magnolia Gardens has a contract with the Texas Department of Human Services (DHS) to provide services, as an assisted living facility, to DHS clients. Gregory Johns--a disabled DHS client who is in a wheelchair, epileptic, and blind--has recently been admitted to Magnolia Gardens and has been charged a small pet deposit for his dog, Courage. Courage is Mr. Johns' emotional support animal. Is Magnolia Gardens in compliance with Chapter 276 of Title 26 of the Texas Administrative Code in regards to charging Gregory Johns a pet deposit? Reply to me right here with a clear answer, and explain your reasoning.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 2/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States Yes, Magnolia Gardens is in compliance with Chapter 276 of Title 26 the Texas Administrative Code as to the pet deposit
PassEvidence: The response says, “Yes — based only on Chapter 276… Magnolia Gardens is likely in compliance by charging Gregory Johns a pet deposit…” and concludes “Magnolia Gardens appears to be in compliance with Chapter 276.” Assessment: Criterion requires stating yes, Magnolia Gardens is in compliance as to the pet deposit. Pass; the response clearly states this, with a reasonable qualification based on Courage being only an ESA.
States that Gregory Johns's dog, Courage, is not a service animal under Chapter 276 of Title 26 of the Texas Administrative Code
PassEvidence: The response says, “If Courage is only an emotional support animal… then he does not fit Chapter 276’s definition of a ‘service animal’,” and applies the facts: “Courage is described as… emotional support animal.” Assessment: Criterion requires stating Courage is not a service animal under Chapter 276. Pass; the response directly states this conditional conclusion on the given facts.
States that Magnolia can charge a pet deposit for pets that are not service animals
PassEvidence: The response says Chapter 276 permits a listed charge of “pet deposit,” except that “A pet deposit does not apply to service animals,” and concludes that if Courage is not a service animal “the pet deposit is permitted.” Assessment: Criterion requires stating Magnolia can charge a pet deposit for pets that are not service animals. Pass; the response clearly conveys that rule.