APEX-Agents · Law
World433_JS_06
APEX-Agents task World433_JS_06 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We are conducting a compliance review of Magnolia Gardens' General Resident Agreement. We are currently determining whether the General Resident Agreement is in compliance with Kentucky Law. Please Review the General Resident Agreement and determine if it is in compliance with Kentucky Revised Statutes (KRS) section 194A.713(7). Please provide a short paragraph back to me here, explaining your reasoning.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that under Kentucky Revised Statues (KRS) section 194A.713(7) the lease agreement must include a minimum thirty-day notice provision for a change in the community's fee structure
PassEvidence: The response says, “The statute requires the lease agreement to include a ‘Minimum thirty (30) day notice provision for a change in the community's fee structure.’” Assessment: This directly states the criterion that KRS 194A.713(7) requires a minimum thirty-day notice provision for changes in fee structure; pass.
States that the General Resident Agreement does not include a minimum thirty-day notice provision for a change in the community's fee structure
PassEvidence: The response says, “I did not find any term requiring at least 30 days’ notice before the community changes its fee structure” and “subsection (7) is missing as drafted.” Assessment: This clearly states the General Resident Agreement lacks the required minimum thirty-day notice provision for a fee-structure change; pass.
States that the General Resident Agreement does not comply with KRS section 194A.713(7)
PassEvidence: The response begins, “Magnolia Gardens’ General Resident Agreement does **not appear compliant** with KRS 194A.713(7).” Assessment: This states that the agreement does not comply with KRS section 194A.713(7); pass.