APEX-Agents · Law
World434_AH_03
APEX-Agents task World434_AH_03 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
One of the Singapore properties we're acquiring (classified PG III) has an exit staircase that services the 1 level underground parking lot, located immediately under the exit staircase that services the above ground floors. The staircases are not continuous and are separated. The audit confirmed that exit distances in the basement to ground level doors comply with the "Determination of Exit Requirement" table. However, the auditor flagged that while the separation between the basement exit staircase and the above-ground exit staircase was fire-rated, the basement exit staircase did not have a fire-rated enclosure and that this was a deficiency that would need to be rectified. The cost will be huge, as the entire building will need to be shut down and major construction completed. Is the auditor's recommendation strictly necessary under Singapore's rules, or can we avoid this cost? Give your assessment to me here as a reply.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| fireworks models Kimi K2 | dual | 2/2 | Pass | Run detailsPublic trace |
| Gemini 3 Flash | dual | 2/2 | Pass | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.4 | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.4 mini | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.5 | dual | 2/2 | Pass | Run detailsPublic trace |
| GPT-5.4 nano | dual | 1/2 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States the auditor's recommendation is incorrect
States a fire-rated enclosure for the basement exit staircase is not required