APEX-Agents · Law
World433_JS_05
APEX-Agents task World433_JS_05 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
We are doing a compliance review to ensure that the General Resident Agreement Template complies with all applicable State laws and regulations. Please review the General Resident Agreement Template and determine if it complies with Kentucky Revised Statutes 194A.713(14). Provide an answer right here in a short paragraph, explaining your reasoning, based on the General Resident Agreement Template.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.4 nano | dual | 6/6 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 4/6 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 4/6 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 4/6 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 3/6 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/6 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that the lease agreement must contain grievance policies that minimally address issues related to confidentiality of complaints under Kentucky Revised Statute (KRS) section 194A.713(14)
States that the lease agreement must contain grievance policies that minimally address issues related to the process for resolving grievances between the resident and the assisted living community under KRS section 194A.713(14)
States that the General Resident Agreement Template does not directly contain grievance policies that minimally address issues related to confidentiality of complaints
States that the General Resident Agreement Template incorporates a resident's rights policy
States that the resident's rights policy has not been provided
States that the General Resident Agreement Template may not comply with KRS section 194.713(14)