APEX-Agents · Law
World431_JS_04
APEX-Agents task World431_JS_04 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
MGR REAL ESTATE INC. (The "Lessor") is leasing 2020 MAIN STREET, IRVINE, CA (The "Premises") to AI AUTOMATION GROUP, LLC (The "Lessee"). The Lessee asked their lawyers to identify provisions that are illegal or unenforceable under California law. Can you list any illegal or unenforceable provisions in Sections 5-8 of the lease agreement. Write back to me in here with your findings.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 5/5 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 1/5 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 0/5 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 0/5 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 0/5 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 0/5 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 0/5 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States that in the "Entry for Repairs and Inspection" clausee, California Civil Code §1954 requires reasonable notice for non-emergency entries
States that in the "Entry for Repairs and Inspection" clause, California Civil Code § 1668 prohibits waiving any remedies arising from the Lessor’s own negligence
States that in the "Indemnification" section, California Civil Code § 1668 disallows the release of all leasor liability to the extent it covers the leasor's own negligence
States that California Civil Code Civil 1654 requires that the contradictory "sole and absolute discretion" and ""may not be unreasonably withheld" standards in the Assignment and Subletting section be interpreted against the party who drafted the contract
States that in the Assignment and Subletting section's Consent Required clause, the "sole and absolute discretion" standard for landlord's consent will be unenforceable per California Civil Code Civil 1654