APEX-Agents · Law
World433_JS_03
APEX-Agents task World433_JS_03 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
Magnolia Gardens provides resident John Marshall with three meals. They are planned out two and a half weeks in advance, as well as two snacks each day. Is Magnolia Gardens in compliance with Part 487 of the New York Codes Rules and Regulations? Give me a clear answer back in here, and concisely explain your reasoning.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| fireworks models Kimi K2 | dual | 3/3 | Pass | Run detailsPublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 3/3 | Pass | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States No, Magnolia Gardens is not in compliance with Part 487 of the New York Codes Rules and Regulations
States that Part 487 of the New York Codes Rules and Regulations requires menus to be planned a minimum of three weeks in advance
States that Magnolia Gardens does not plan menus at least three weeks in advance