APEX-Agents · Law
World434_AH_04
APEX-Agents task World434_AH_04 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
We had an intern prepare a list of what needed to be done after closing and they made a mess of it. We've already established the holding company, reviewed insurance policies, and completed a compliance & safety audit. Several of the other items noted need to be completed pre-closing. Identify each such item from the list - and do not include optional items, the local leads will work on these once closing is completed, we just need mission critical items identified. Reply here with your findings in a short message that gives me everything I asked for.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 4/4 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 4/4 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/4 | Fail | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 2/4 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 1/4 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 1/4 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 0/4 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States any part of the following item from the pre-closing check-list needs to be completed prior to closing: "upgrade or purchase robust property-insurance, general liability, fire & hazard insurance, business interruption, environmental liability coverage, hazardous-materials coverage (if feasible), third-party liability”
States any part of the following item from the pre-closing check-list needs to be completed prior to closing: "define board structure”
States any part of the following item from the pre-closing check-list needs to be completed prior to closing: "appoint local management leads for HK and SG”
States any part of the following item from the pre-closing check-list needs to be completed prior to closing: "maintain a contingency reserve / CapEx fund for ongoing expected maintenance, compliance updates, retrofits, permit renewals, safety upgrades, unforeseen regulatory changes”