Raycaster / evalsTask overviewAPEX-AgentsCategory

APEX-Agents · GPT-5.5 · dual

Law 433_AC_02

4/10Fail

GPT-5.5 on APEX-Agents: Law 433_AC_02 (dual harness). Browse score, rubric, and public trace.

4/10 · Fail
Law
AI Agents for Healthcare and Senior Living Legal Risk
Law World 433

Grader rubric

Criteria verdict

  1. States that the Resident Rights and Dignity section was removed from the 2025 manual

    Fail
  2. States that the 2025 manual may violate the regulatory requirement for Grove to advise residents of their rights

    Fail
  3. States that the 2025 manual only requires a registered nurse to be on-call at night

    Fail
  4. States that the 2025 manual may violate the regulatory requirement for Grove to staff registered nurses on-site to conduct overnight duties

    Fail
  5. States that the 2025 manual removed dietician approval for meals offered to residents

    Pass
  6. States that the 2025 manual may violate the regulatory requirement for Grove to maintain quality control over dietary services

    Fail
  7. States that the 2025 manual allows for non-urgent complaints to be acknowledged within 14 days

    Pass
  8. States that the 2025 manual may violate the regulatory requirement for Grove to respond promptly to complaints

    Pass
  9. States that the Emergency Preparedness section was removed in the 2025 manual

    Fail
  10. States that 2025 manual may violate the regulatory requirement for Grove to develop and maintain a written emergency preparedness and response plan

    Pass

Prompt excerpt

Task context

Please review Grove’s 2023 and 2025 operations manuals. Let me know if there are any changes in the 2025 manual that may present issues for Grove’s regulatory compliance. For any regulatory issues identified, tell me what changed in the 2025 manual and why it presents an issue. Reply back just with one or two sentences please.

Response trace

Agent response, tools, files, and edits

Open full trace

On mobile, open the full trace for the best view of the agent response, tool calls, file reads, edits, and grading context.

Open mobile trace