APEX-Agents · Law
World434_IG_02
APEX-Agents task World434_IG_02 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models, scores, and public traces.
Task prompt
What the agent was asked to do
One of the Hong Kong facilities is going to overhaul its fire safety after they found that some of the equipment from the supplier produced faulty sprinklers. This same supplier produced essentially all of the equipment that can't be used. The facility needs 50 fire doors to be installed by code. Most doors are very expensive, but we have identified three in our price point. Assuming the dimensions are proper, please state the cheapest one that we can purchase and use. Explain your reasoning. Reply to me here concisely.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| Gemini 3 Flash | dual | 4/4 | Pass | Run detailsPublic trace |
| fireworks models Kimi K2 | dual | 3/4 | Fail | Run detailsPublic trace |
| Gemini 3.1 Pro | dual | 3/4 | Fail | Run detailsPublic trace |
| GPT-5.4 | dual | 3/4 | Fail | Run detailsPublic trace |
| GPT-5.5 | dual | 3/4 | Fail | Run detailsPublic trace |
| GPT-5.4 mini | dual | 1/4 | Fail | Run detailsPublic trace |
| GPT-5.4 nano | dual | 0/4 | Fail | Run detailsPublic trace |
Grading rubric
Rubric criteria
Runs are graded against these criteria. Open a run for model-specific verdicts.
States the AegisCore door cannot be sourced for the Hong Kong facilities
States the SentinelGuard does cannot be sourced for the Hong Kong facilities
States that the Fireline Veritas doors comply with Hong Kong fire safety regulations
States the Fireline Veritas doors are the cheapest doors which should be sourced for the Hong Kong facilities